A student experiment on error analysis and uncertainties based on mobile--device sensors
Martin Monteiro, Cecilia Stari, Cecilia Cabeza, Arturo C. Marti
AA student experiment on error analysis and uncertainties basedon mobile–device sensors
Mart´ın Monteiro ∗ Universidad ORT Uruguay
Cecila Stari, Cecila Cabeza, and Arturo C. Mart´ı † Instituto de F´ısica, Facultad de Ciencias,Universidad de la Rep´ublica, Igu´a 4225, Montevideo, 11200, Uruguay (Dated: June 3, 2020)
Abstract
Science students must deal with the errors inherent to all physical measurements and be consciousof the necessity to express their as a best estimate and a range of uncertainty. Errors are routinelyclassified as statistical or systematic. Although statistical errors are usually dealt with in thefirst years of science studies, the typical approaches are based on performing manually repetitiveobservations. Here, based on data recorded with the sensors present in many mobile devices a set oflaboratory experiments to teach error and uncertainties is proposed. The main aspects addressedare the physical meaning of the mean value and standard deviation, and the interpretation ofhistograms and distributions. Other activities focus on the intensity of the fluctuations in differentsituations, such as placing the device on a table or held in the hand in different ways and thenumber of measurements in an interval centered on the mean value as a function of the widthexpressed in terms of the standard deviation. As applications to every day situations we discussthe smoothness of a road or the different positions to take photographs both of them quantified interms of the fluctuations registered by the accelerometer. This kind of experiments contributes togaining a deep insight into modern technologies and statistical errors and, finally, to motivate andencourage engineering and science students. a r X i v : . [ phy s i c s . e d - ph ] J un . INTRODUCTION In many experimental situations when a measurement is repeated, for example when wemeasure a time interval with a stopwatch, or the distance at which a ball launched witha spring-loaded projectile launcher falls or a voltage with a digital multimeter, successivereadings, under identical conditions, give slightly different results. This occurs beyond thecare we take to always launch the balls in exactly the same way or to connect the componentsof the circuit so that they are firmly attached. In effect, this phenomenon is due to thefact that most measurements in the real world present statistical uncertainties . Whenfacing repeated observations with different results it is natural to ask ourselves what is themost representative value and what is the confidence that we can have in that value. TheInternational Standard Organization (ISO) defines the errors evaluated by means of thestatistical analysis of a series of observations as type A in contrast with other, systematic ,sources of errors, type B, whose evaluation is estimated using all available non-statisticalinformation such as instrument characteristics or observer’s individual judgment. In thiswork, we focus on the teaching of statistical errors in the first years of engineering andscience studies using modern sensors.The study of error analysis and uncertainties plays a prominent role in the first years ofall science courses. Perhaps the most important message is to persuade students that anymeasurement is useless unless a confidence interval is specified. It is expected that afterfinishing their studies, students are able to discuss whether a result agrees with a given the-ory, or if it is reproducible, or to distinguish a new phenomenon from other already known.With this objective, various experiments are usually proposed in introductory laboratorycourses . These experiments usually involve a great amount of repetitive measurementssuch as dropping small balls or measuring the length of hundred or thousands of nailsusing a vernier caliper . The measurements obtained are usually examined from the statis-tical viewpoint plotting histograms, calculating mean values and standard deviations and,eventually, compared with those expected from a known distribution, typically a normaldistribution. Though these experiments are illustrative, most of them are tedious and donot adequately reflect the present state of the art.The importance for their careers of a physicist being able to design a measurement proce-dure, select the equipment or instruments, perform the process and finally express the results2s the best estimation and its uncertainties has been remarked. However, recent studies ,suggest that students lack these abilities. Several difficulties have been described : the lackof understanding of the need to make several measurements, or insight into the notion ofconfidence interval or the ability to distinguish between random and systematic errors.Mobile devices such as smartphones or tablets which usually include several sensors (ac-celerometer, magnetometer, ambient light sensor, among others) appear as modern andversatile alternatives to deal with statistical errors. In fact, the use of smartphone sen-sors has been proposed in many science experiments , ranging from experiments withquadcopters to shadowgraph imaging . The inevitable noise of the sensors, so annoyingin any measurement, can be used, however, favorably, to illustrate basic concepts of statis-tical treatment of measurements. It is possible, using these sensors, to acquire hundreds orthousands of repeated values of physical magnitude in a few seconds that can be analyzedin the mobile device or transferred to a PC. Thanks to their sensitivity these values clearlydisplay statistical fluctuations. In this paper we propose a set of laboratory activities toteach error analysis and uncertainties using modern technologies in a stimulating approach.In the next Section we review some basic concepts about error analysis, while in Section IIIwe describe the proposed activities. Finally, in Section IV we present the summary andconclusion. II. STATISTICAL ERRORS
In this work we focus on the teaching of statistical errors which due to a multitude ofcauses are inherent to all physical measurements . We assume that in a given experimentan observation is repeated N times under identical conditions obtaining different results x i ,with i = 1 , .., N . It can be shown that the best representative or estimate of the set of valuesis given by the mean value x defined as x = 1 N N (cid:88) i =1 x i . (1)The deviation with respect to the mean value is identified with (cid:15) i = x i − x . It can be shownthat the mean value defined as above minimizes the sum of the squared deviations. Intu-itively, it can be regarded as the center-of-mass of the set of the observations or equivalentto the value closest to all the other values. In statistical errors it is of interest to quantify3he dispersion of the values around the mean value or, informally, the width of the cloud .The standard deviation defined as σ = (cid:118)(cid:117)(cid:117)(cid:116) N − N (cid:88) i =1 ( x i − x ) (2)can be seen as a measure of this dispersion. If the number of observations, N , is largeenough, σ it is characteristic of the set of all the possible observations and does not dependon the specific set of observations. In practice, the uncertainty in the determination of aphysical magnitude depends on the number of repeated measurements we have done.The standard error, or standard deviation of the mean, is defined as σ x = σ/ √ N and itis demonstrated that it represents the margin of uncertainty of the mean value obtained ina particular set of measurements. The result of a specific measurement is usually expressedin terms of the mean value and the standard error as x ± σ x (3)representing the best estimate and the confidence in that value. It is worth highlighting thatthe standard deviation is related to the degree to which an observation deviated from themean value whereas the standard error is an estimate of the uncertainty of the mean value. Ina practical situation the standard error depends on the number of measurements taken with N − / . Then, given a set of N measures the standard deviation gives an idea of the dispersionof an ideal set of infinite measures while the standard error represents the uncertainty of ourset. This margin can be reduced by increasing the number of measurements, however, thesquare-root implies that this reduction is relatively slow.It is an empirical fact that when the uncertainties of a continuous magnitude do nothave a preferred direction they follow a normal or Gaussian distribution. The probabilitydistribution function resembles the well-known bell-shaped curve centered around the meanvalue observed in many phenomena in natural and social sciences. The width of the bell isgiven by the standard deviation, the inflection points are located at x ± σ . III. A LABORATORY BASED ON MOBILE DEVICES
The vast majority of smartphones and tablets have several built-in sensors, in particular,triaxial accelerometers capable of measuring the acceleration of the device in the three inde-4endent spatial directions. Though it is possible to use all the components simultaneously,here, for the sake of clarity, the following experiments are based on the z direction whichis defined as perpendicular to the screen. To access the values registered by the sensors aspecific piece of software or app is necessary.From the many apps available in the digital stores we selected Physics Toolbox Suite ,Androsensor and PhyPhox whose screenshots are shown in Fig. 1. Using these apps it ispossible to select the relevant sensors, and to setup the parameters such as the duration ofthe time series and the sampling frequency. The registered data can be analyzed directlyon the smartphone screen or transferred to the cloud and studied on a Personal Computerusing a standard graphics package. Others useful characteristics present in these apps arethe delayed execution and the remote access via wi-fi or browser. These capabilities allowthe avoidance of touching or pushing the mobile device when the experiments has started. FIG. 1. Screenshots of the most used apps : Physics Toolbox suite (left) and Phyphox (center andright). The right panel shows a Phyphox screenshot of the experiment
Statistical Basics including atemporal series of the vertical component of the acceleration (top) and the corresponding histogram(bottom) overlapped with a Gaussian curve with the same mean and standard deviation indicatedin the image. . A first approach to fluctuations The first experiment consists of recording the fluctuations of the vertical component ofthe accelerometer sensor with the mobile device standing on a table, during a time lapse. Inthis experiment, and all the described above, it is possible to use an app and download thedata or use the PhyPhox app or to choose in the menu the experiment
Statistical physics which automatically displays temporal series and histograms. In our case, we choose, unlessstated otherwise, a delay of 3 s and register a z for 10s. The 3 s delay is important to avoidtouching the device at the moment the register starts and introducing spurious values. Thescreenshot is displayed in Fig. 1 (right). In this case the number of measurements and thesampling period are N = 2501 and ∆ t = 0 .
004 respectively.Although the device is at rest on a horizontal surface, the a z values displayed in Fig. 2fluctuate steadily around a mean value given by the gravitational acceleration x = 9 , and a standard deviation σ = 0 . . The non-zero mean value is due to the fact thataccelerometers are in fact force sensors that cannot distinguish between the accelerationand the gravitational field . If, instead of the acceleration sensor, the so-called linear acceleration pseudo-sensor were used, the measurements would fluctuate around 0 m/s .The corresponding histogram is displayed in Fig. 3 with, for the sake of comparison, anormal (Gaussian) curve with the same mean value and standard deviation. The verticalscale has been adjusted so that the area under the normal curve and the sum of the bins of thehistogram are both equal to 1. From this figure, it can be concluded that the histogram andthe normal curve agree very well. By increasing the number of samples N and simultaneouslydecreasing the width of the bins, it can be seen (not shown here) that the agreement improveseven more. B. Resolution in digital sensors
It can be noticed in Fig. 2 that the sensor values display a clear regularity, the ordinatesdo not take arbitrary continuous values but only a discrete set. This is more evident inFig. 4 where, in the left panel, the horizontal axis of Fig. 2 has been zoomed out and,in the right panel, a layed down histogram with the same values is shown. The differencebetween the discrete values in the vertical axis is the resolution of the instrument, that is, the6
IG. 2. Temporal values of the z component of the accelerometer while the smartphone is standingat rest on a table. The values registered by the sensor are indicated with small circles while thelines are guides for the eye. minimum difference that the sensor can register. This is typical of digital instruments, wherea continuous magnitude (such as acceleration, in this case) is transformed by a sensor into ananalog electrical signal, which is transformed by an analog-to-digital converter (ADC) intoa digital signal which can only take certain discrete values. The acceleration sensor of theSamsung S7 is a K6DS3TR, as shown in Table I. The resolution given by the manufacturer(sometines it appears incorrectly as accuracy), is δ = 0 . , which, as canbe seen in Fig. 4, corresponds exactly to the difference between the groups of accelerationvalues.The resolution of the sensor can be related to other important characteristic of the digitalsensors. One is the range of the sensor, R , corresponding to the difference between themaximum and minimum value that it is capable of measuring. The maximum number ofdifferent values that the sensor can register is 2 n where the n is known as the number of bitsof the DAC. Resolution is simply the quotient between the range and the total number of7 IG. 3. Histogram of the values from Fig. 2 and a Gaussian curve with the same mean value,standard deviation and normalization different values, that is, δ = R n . (4)In the sensor used in this experiment Table I shows that the accelerometer used in this casecan measure a maximum acceleration of 78 . . Since it registers not only positivemeasures, but also negative accelerations, the range turns out to be twice the maximumvalue, that is, R = 156 . . Therefore it can be determined that this sensor iscapable of measuring R/δ = 65536 different values and since 65536 = 2 , this means thatit is a 16-bit sensor. These characteristics can be easily verified on the datasheets of thesensors. C. Different noise intensities
In order to gain insight into the role of noise in different situations in this experimenttwo sets of data are considered. In the first the smartphone is steady on a table and in the8
IG. 4. Discrete nature of the sensor data. The left panel is similar to Fig.2 but zoomed out inthe horizontal axis to emphasize the discrete nature of the accelerometer values. The right panelshows the same values in a layed down histogram with the same vertical scale. other the device is held in the hand of the experimenter. In Fig. 5 both temporal series areshown while in Fig. 6 the corresponding histograms are displayed. Moreover, histograms areoverlapped with normal curves with their respective mean values and standard deviations.It is clearly appreciated that the dispersion of data, quantified by the standard deviation,is larger when the smartphone is held in the hands than when the device is on the table. Itis also noticeable in both cases that normal curves agree very well with the histograms. Thisactivity can be translated to other settings. In particular, this is one the basic mechanismsof seismographs.
D. Number of observations in a given interval
In general, the fundamental property of distributions is that the area under one sector ofthe curve represents the probability that a new measurement falls within this interval. In9 hone Sensor Range (m/s ) Resolution (m/s )Samsung Galaxy S7 K6DS3TR 78,4532 0,0023942017LG G3 LGE 39,226593 0,0011901855Nexus 5 MPU-6515 19,613297 0,0005950928iPhone 6 MPU-6700Samsung J6+ LSM6DSL 39,2266 0,0011971008Xiaomi Redmi Note7 ICM20607 78,4532 0,0011901855Samsung Galaxy S9 LSM6DSL 78,4532 0,0023942017TABLE I. Sensor characteristics of the devices used in the different activities obtained with theAndrosensor app. In the case of the iPhone the manufacturer does not provides this information. the case of normal curves, it is usual to take intervals centered around the mean value andthe width in terms of the standard deviation. Then, it is shown that 68% of the observationswill be in the ” σ ” interval, this is the interval between x − σ and x + σ , P ( x − σ < x < x + σ ) = (cid:90) x + σx − σ f ( x ) dx = 0 . ... (5)Similarly, the intervals ”2 σ ,” ”3 σ ,” and ”4 σ ” concentrate, respectively, 95.4%, 99.7%, and99.9% of the observations. This is a characteristic of normal distributions, i.e. , almost all theobservations are concentrated around a few ”sigmas” around the mean value and graphicallythe curve is relatively stretched.To illustrate this phenomenon, Fig. 7 shows the temporal series of Fig. 2 with horizon-tal lines indicating the σ intervals. It is evident from this figure that most of the valuesconcentrate around the mean value and a few σ intervals. To quantify this relation, twoexperiments with different noise intensities (on the floor and on an aircraft) are describedin Table II. In this table the number of observations in a given interval are compared withthe expected values according to the normal distribution. It can be seen that the expectedpercentages are similar to those according to a normal distribution.An interesting point is to express these ranges in terms of the resolution of the sensor.In this way it is noted that 68% of the measurements are within a radius interval equal to3 times the resolution. On the other hand, 99% of the measurements are within a radiusinterval equal to 10 times the resolution of the sensor.10 IG. 5. Temporal series of the acceleration values on the two different situations: first, the mobiledevice is on a table (blue lines) and secondly, held in a hand (red lines).
E. Optimal number of measurements
Accuracy and precision are different concepts . On the one hand, the precision of a mea-surement, related to the random errors, is characterized by the dispersion of the values, i.e.the standard deviation. The smaller σ , the less dispersion and therefore, the greater theprecision. On the other hand, accuracy is related to systematic error and quantified accord-ing to the agreement with an expected value. As mentioned above, in observations underidentical and independent conditions, the standard deviation does not change considerablywith the number of observations N . In contrast, the standard error, giving account of therange of confidence in the estimation of the mean value in a particular set of measurementsdecreases with N − / . In Table. III the standard deviation and the standard error are shownfor different set of observations with different N . It is clear from that data, as mentionedabove, σ is nearly constant while σ x clearly decreases.As the decrease of the standard error with N is slow, an important question in practical11 IG. 6. Comparison between different noise intensities with the mobile device steady on a table(blue) or held in a hand (red). Data is the same displayed in the temporal series of Fig. 5.Continuous lines are Gaussian curves with the same mean values ( x blue = 9 .
474 m/s and x red =9 .
362 m/s ), standard deviations ( σ blue = 0 .
019 m/s and σ red = 0 .
066 m/s ) and normalizationfactor corresponding respectively to the histograms of the same color. situations is about the optimal number of observations N opt . Indeed, if we could repeatthe measurements infinite times we could achieve a perfect knowledge of the best estimateand, accordingly, the standard error would vanish. In fact, in addition to the statisticalerrors, type B errors must be taken into account. In absence of other sources of systematicerrors, the optimal number of observations is defined when the standard error is equal tothe resolution of the digital instrument σ x = σ/ √ N ∼ δ . In the experiment, depicted inTable III, given the resolution of S7 model 0 . , the optimal number is N opt ∼ . IG. 7. Temporal series indicating the vertical intervals in term of units of the standard deviation σ . F. The best position to take a photograph with a cell phone
An interesting experiment is to study the intensities of the fluctuations depending onthe way in which the experimenter holds his/her device. This activity can be adapted tobe proposed as a challenge to a group of students consisting in trying to hold the deviceas steadily as possible. Another possibility, not recommended by the authors, is, similarlyto Ref. , to study the fluctuations of the gait of a pedestrian as a function of the alcoholbeverage intake.The steadiness of the device is quantified by the standard deviation of a given temporalseries. In Table IV we display the intensities of the fluctuations in different positions. It isevident from these values that keeping the device close to trunk represents a more stableposition. 13 xperiment 1, N = 2098, a z = (9 . ± . Interval Theo. (%) Theo. Exp.(%) Exp. x ± σ x ± σ x ± σ N = 1501, a z = (9 . ± .
29) m/s Interval Theo. (%) Theo. Exp.(%) Exp. x ± σ x ± σ x ± σ G. The smartphone as a way to assess road quality
Recently, smartphones’ sensors were proposed to assess road quality . In this activity,which can be performed outdoors, students can assess the quality of a road. A meansof transport, in this case a car, under similar conditions (speed) is employed, but otherpossibilities, such as a bike, are equally feasible. In Table V the intensities of the fluctuationstraveling by car on different roads are listed. To get an insight of the fluctuations due tothe road in the first row the noise with the car stopped and engine idle is indicated. Justfor the sake of comparison a similar measurement but in a flying aircraft is included.The intensity of the fluctuations depends on the specific sensor but exhibits in all casesthe same trends mentioned above. To summarize the results, all the intensities of thefluctuations using the different built-in sensors in several situations are depicted in Fig. 6. IV. CONCLUSION
The main conclusion is that modern mobile-device sensors are useful tools for teachingerror analysis and uncertainties. In this work we proposed several activities that can beperformed to teach uncertainties and error analysis using digital instruments and the builtin14 g σ σ x
563 9,493 0,020 0,000851156 9,487 0,019 0,000541746 9,478 0,018 0,000442348 9,469 0,019 0,000392941 9,466 0,020 0,000363535 9,464 0,019 0,000324166 9,462 0,019 0,000294733 9,464 0,019 0,000275327 9,465 0,019 0,000265919 9,464 0,020 0,00026TABLE III. Mean value, standard deviation and standard error as a function of the number ofmeasures.FIG. 8. Comparative table of the standard deviation σ for different mobile devices in differentactivities as a function of the different models (see Table I). martphone position N σ G3 (m/s ) N σ XR7 (m/s )On the table 1746 0.0184 2407 0.0052Close to the body 1190 0.067 2502 0.1030Selfie position 1190 0.1206 2512 0.1413TABLE IV. Standard deviation of a z of two smartphone models in different positions.Situation N σ G3 (m/s ) N σ XR7 (m/s )Engine idle 1181 0.3818 4984 0.0352Smooth pavement 1200 1.3487 4974 0.5642Stone pavement - - 4952 1.1491Aircraft 1999 0.4374 - -TABLE V. Assessment of the quality of different roads. Standard deviation of a z while the deviceis on the floor of the car with the screen orientated upwards. sensors included in modern mobile devices. It is shown that the distribution of fluctuationsobeys normal (Gaussian) statistics. Its main characteristics –mean, standard deviation,histograms– are analyzed. The role of noise intensity, spreading or narrowing the normalbell-shapped curve is revealed. The width of the distribution in terms of units of the standarddeviation can be related to the number of measurements in a given interval. Holding themobile in different ways also gives an idea of how firmly it is held. In this approach, thelengthy and laborious manipulations necessary in traditional approaches based on repetitivemeasurements, are avoided allowing teaching to focus on the fundamental concepts. Theseexperiments could contribute to motivating students and to showing them the necessity ofconsidering uncertainty analysis. 16 CKNOWLEDGMENT
The authors would like to thank grant Fisica Nolineal (ID 722) Programa Grupos I+DCSIC 2018 (UdelaR, Uruguay). ∗ [email protected] † marti@fisica.edu.uy John Taylor.
Introduction to error analysis, the study of uncertainties in physical measurements .University Science Books, 1997. Ifan Hughes and Thomas Hase.
Measurements and their uncertainties: a practical guide tomodern error analysis . Oxford University Press, 2010. IEC ISO and BIPM OIML. Guide to the expression of uncertainty in measurement.
Geneva,Switzerland , 122, 1995. I. M. Meth and L. Rosenthal. An experimental approach to the teaching of the theory ofmeasurement errors.
IEEE Transactions on Education , 9(3):142–148, 1966. E Mathieson and TJ Harris. A student experiment on counting statistics.
American Journal ofPhysics , 38(10):1261–1262, 1970. PCB Fernando. Experiment in elementary statistics.
American Journal of Physics , 44(5):460–463, 1976. Tadeusz Wibig and Punsiri Dam-o. ‘hands-on statistics’—empirical introduction to measure-ment uncertainty.
Physics Education , 48(2):159, 2013. KK Gan. A simple demonstration of the central limit theorem by dropping balls onto a grid ofpins.
European Journal of Physics , 34(3):689, 2013. Marie-Genevi`eve S´er´e, Roger Journeaux, and Claudine Larcher. Learning the statistical analysisof measurement errors.
International Journal of Science Education , 15(4):427–438, 1993. Saalih Allie, Andy Buffler, Bob Campbell, Fred Lubben, Dimitris Evangelinos, Dimitris Psillos,and Odysseas Valassiades. Teaching measurement in the introductory physics laboratory.
ThePhysics Teacher , 41(7):394–401, 2003. MF Chimeno, MA Gonzalez, and J RAMOS Castro. Teaching measurement uncertainty toundergraduate electronic instrumentation students.
International Journal of Engineering Edu- ation , 21(3):525–533, 2005. Rebecca Vieyra, Chrystian Vieyra, Philippe Jeanjacquot, Arturo Marti, and Mart´ın Monteiro.Five challenges that use mobile devices to collect and analyze data in physics.
The ScienceTeacher , 82(9):32–40, 2015. Katrin Hochberg, Jochen Kuhn, and Andreas M¨uller. Using smartphones as experimentaltools—effects on interest, curiosity, and learning in physics education.
Journal of Science Edu-cation and Technology , 27(5):385–403, 2018. Martin Monteiro and Arturo C Marti. Using smartphone pressure sensors to measure verticalvelocities of elevators, stairways, and drones.
Physics Education , 52(1):015010, 2017. Gary S. Settles. Smartphone schlieren and shadowgraph imaging.
Optics and Lasers in Engi-neering , 104:9 – 21, 2018. Optical Tools for Metrology, Imaging and Diagnostics. Rebecca Vieyra and Chrystian Vieyra. Physics toolbox suite, July 2019. S Staacks, S H¨utz, H Heinke, and C Stampfer. Advanced tools for smartphone-based experi-ments: phyphox.
Physics Education , 53(4):045009, may 2018. Mart´ın Monteiro, Cecilia Cabeza, and Arturo C Mart´ı. Exploring phase space using smartphoneacceleration and rotation sensors simultaneously.
European Journal of Physics , 35(4):045013,2014. Mart´ın Monteiro, Cecilia Cabeza, and Arturo C. Marti. Acceleration measurements usingsmartphone sensors: Dealing with the equivalence principle.
Revista Brasileira de Ensino deF´ısica , 37:1303 –, 03 2015. Tirra Hanin Mohd Zaki, Musab Sahrim, Juliza Jamaludin, Sharma Rao Balakrishnan,Lily Hanefarezan Asbulah, and Filzah Syairah Hussin. The study of drunken abnormal hu-man gait recognition using accelerometer and gyroscope sensors in mobile application. In , pages151–156. IEEE, 2020. PM Harikrishnan and Varun P Gopi. Vehicle vibration signal processing for road surface mon-itoring.
IEEE Sensors Journal , 17(16):5192–5197, 2017., 17(16):5192–5197, 2017.