An HTM based cortical algorithm for detection of seismic waves
AAn HTM based cortical algorithm for detection of seismicwaves.
Ruggero Micheletto , Ahyi Kim Abstract
Recognizing seismic waves immediately is very important for the realization of efficientdisaster prevention. Generally these systems consist of a network of seismic detectorsthat send real time data to a central server. The server elaborates the data and attemptsto recognize the first signs of an earthquake. The current problem with this approach isthat it is subject to false alarms. A critical trade-off exists between sensitivity of thesystem and error rate. To overcame this problems, an artificial neural network basedintelligent learning systems can be used. However, conventional supervised ANN systemsare difficult to train, CPU intensive and prone to false alarms. To surpass these problems,here we attempt to use a next-generation unsupervised cortical algorithm HTM [5]. Thisnovel approach does not learn particular waveforms, but adapts to continuously fed datareaching the ability to discriminate between normality (seismic sensor background noisein no-earthquake conditions) and anomaly (sensor response to a jitter or an earthquake).Main goal of this study is test the ability of the HTM algorithm to be used to signalearthquakes automatically in a feasible disaster prevention system. We describe themethodology used and give the first qualitative assessments of the recognition ability ofthe system. Our preliminary results show that the cortical algorithm used is very robustto noise and that can successfully recognize synthetic earthquake-like signals efficientlyand reliably.
Introduction
This is a brief report on the setup of a Hierarchical Temporal Memory (HTM) corticalmodel for real-time detection of seismic waves. Generally, early detection of dangerousseismic phenomena can be done by hybrid methodology, like on-line analysis of amplitude,duration and spectrum of the vibration [1]. However, this approach is subject to a highrate of misses and false alarm. Errors are caused by human induced spurious signalsdue to automotive traffic, local mechanical disturbances or other phenomena. To reducethem it is necessary to distinguish between these unwanted signals and real seismic wavesembedded in noise. This can be done with an intelligent software able to learn the shapeof a natural seismic and recognize it from a background of noise and human-induceddisturbances. For this purpose, artificial neural networks have been proposed. Thesenetworks are able, if properly trained by trains of data with examples of real seismicwaves and noises, to classify wave shapes and deduce if they are dangerous seismic wavesor innocuous artificial disturbances. However, training of these networks is difficultand precision of classification is still low [2–4]. To overcame these problems, we test1/7 a r X i v : . [ c s . N E ] J u l igure 1. Top panel: the synthetic seismic waveform fed to the HTM model. This ismade by a uniform random acceleration signal between -1 and 1 and sporadic seismicwaves . These are synthesized by equation 1 with a probability p = 0 . t = 400, t = 600 and t = 750) fromthe simulated background noise. Time t is an arbitrary sample interval here, in a realexperiment this interval will be the integration time of the seismic sensor. 2/7ere an anomaly-detection unsupervised algorithm based on biological plausible corticalstructure. These models are developed on a new theoretical framework that constructa hierarchical temporal memory system able to learn a sequence of data that is fedcontinuously to the input. These HTM systems perform similarly to other sequencelearning algorithms like autoregressive moving averages, feedforward neural networksand recurrent neural networks, but have the advantage to be ready for continuous onlinelearning and are inherently specialized for anomaly detection.The seismic sensors will feed continuously data to the network. The data are synthetic,generated by an algorithm in order to represent a combination of instrumental noiseand human generated spurious signals (people walking in the room nearby the sensor,slamming doors, traffic nearby etcetera). The cortical algorithm is fed continuouslywith this data that will be learned as the normality by the system. Our hope is thatdeviation from this normality (an earthquake) will result in an anomaly signal by theHTM. This approach has the advantage that we do not need training, the network willlearn by itself what is normal and what it is an anomaly in a unsupervised way. Sincewe plan to realize a network of sensors placed in disparate places, an HTM network willlearn specific local disturbances, whereas standard supervised methods may need tobe re-calibrated for sensors located in different places. An HTM network has also theability to predict future evolution of the signal, it is robust to sensor noise, has faulttolerance and exhibit good performance without the need for task-specific tuning [5]. Figure 2.
After about t = 450 .
000 time steps (plot shows a window of 1200 of them),the network lowers its anomaly output, and gains an impressive robustness to noise. Theanomaly score drops cleanly to zero when random noise is constantly fed to the model.This results in distinct identification of the simulated seismic waves visible in the signalof the top panel at t ≈
50. Moreover, small waveforms, barely distinguishable by eye,give rise to net anomaly score around t = 500 in the bottom panel. Method
For convenience seismic waves were synthesized by a simple algorithm. This gives us theadvantage to experiment with the HTM network with calibrated and optimal data verysimilar to the expected real case scenario signals. We didn’t use real sensors data in thiscase, because those should be placed in a noisy laboratory where all sort of activities areon going. This in practice will limit the amount of usable data for our tests. 3/7he algorithm basically simulate both instrumental noise and small jitters thatrepresent human noises and vibrations in real experimental environments. The instru-mental noise is simulated with a simple uniform distribution of numbers withing theinterval − ,
1. These values are representing normalized accelerations. Jitters are insteadoccurring with a probability of p = 0 .
005 (on average once every 200 points) and aregenerate summing up several random sine waves. Overall the signal is obtained withthis formula: A c ( t ) = Σ n asin (2 πf n t ) + (cid:15) (1)where f n are ten frequencies chosen at random in the interval f min = 0 .
01 and f max = 0 . (cid:15) is the uniformly distributed instrumental noise mentioned above. Thejitter duration is fixed to 25 time units and its amplitude a is a random number between0 to five.The calculations are done by a 64 bits Linux (Ubuntu) machine with 6 Gb Ram and4 threads CPU. Numenta NuPIC was installed on the system [7] to setup the HTMcortical network. NuPIC is a package that implements the HTM network structure, theuser can manipulate the parameters that regulate its functioning.To determine the best configuration for our HTM network we did a parameter searchthrough a swarming process [6]. The final relevant parameters of our cortical networkare the one in table 1. Notice that those numbers are given in NuPic’s conventionalorder and naming. To understand exactly the meaning of the table, readers shouldrefer to HTM literature (see provided literature about HTM and NuPic [5, 7]). Usage ofNuPic HTM cortical algorithms will generate parameters that can compare directly tothis table. Table 1. Cortical Algorithm Parameters
The parameters used by the HTM corticalalgorithm. This table is given as a guideline. Sp and tp parameters are give in theHTM conventional order. Readers should learn details about the functioning of HTMalgorithm to understand fully the meaning of the values.
HTM parameters alpha = 0.009340 SDRClassifierRegion steps = 1Verbosity = 0 inference = anomaly sensor parameters clip = True max = 2.0 min = -2.0n= 118 type = scalarEncoder w = 21 sp parameters tp parameters
12, 32, 2048, 0, 0.21, 2048, 0, 128, 32, 9, 20, normal, 1, 0.1, 0.1, 1960, cpp, 0
Once the network is correctly setup with the parameters in Tab (1), the model is run.We process an infinite loop in which synthetic seismic noise and waveforms are generatedby expression 1, each generated value at each time-step is fed to the model that usesthe multiStepBestPredictions method to attempt prediction of the next accelerationvalue. Also the model outputs an anomalyScore value ranging from 0 to one. This valuerepresents how the model feels about the current signal behavior, if it is consideredhighly anomalous, the score will be high, otherwise will be low. The anomaly scoreranges from zero to one. 4/7 igure 3.
The acceleration signal (top) and the corresponding HTM model predictedvalue. The HTM algorithm predictions are made one step ahead and are similar to inputvalues.In figure fig. 1 are shown the first few thousands time steps of the model simulation.Each time steps corresponds to an arbitrary unity of time, in an experiment run with areal seismic sensor this time-step will correspond to the device integration time.Clearly the model is not able at this stage to discriminate between noise and anomalousseismic waves. Those are represented in fig. 1 top panel at about t = 600 and t = 750.The cortical algorithm considers random noise as an anomaly and other waveforms too.This behavior makes sense since there is nothing more unpredictable and anomalous than a random variable and other waveforms are new and unpredictable as well.However, if we let the system run for many thousands cycles, the model begins toadapt to the continuous random noise, and eventually the anomaly score drops stably tozero each time a sequence of noise arrives. This shows how the HTM cortical algorithmhas adapted to the random signal continuously fed to it. This somewhat compares tohuman or animal behavior when external disturbances are ignored if they are repeatedregularly. In figure 2 lower panel we see the response of the algorithm to the randomseismic signal after about half a million time steps. As shown in the lower panel, themodel anomaly score to pure noise keeps nearly zero in a robust and reproducible manner.Interestingly, the simulated seismic waves instead are recognized as anomalous. Theseismic signals visible in the top panel are correctly identified with higher anomaly scores.These fluctuations have their characteristics and are much less common than the randomnoise, nevertheless, after the adaptation period, the model becomes able to distinguishsuccessfully between the two. Noticeably the duration of anomalous response is slightlydelayed compared to the seismic waves. For example the short perturbation at about t = 500 in the top panel corresponds to a longer anomaly score response. This particularwaveform is very small and barely recognizable by eye, nevertheless the cortical algorithmwas able to distinguish it from the background noise. This is an outstanding behaviorthat may prove to be useful for the realization of a disaster prevention system.Simultaneously to the anomaly scores, the model outputs a prediction of the nextacceleration value, basing itself to previous accelerations sequences. In figure 3 weshow a plot of the acceleration signal and the corresponding predictions. Interestinglythe prediction signal looks very similar to the actual waveform, but it precedes thewaveform output by one steps. The system is able to somehow anticipate future seismic5/7ccelerations, this could be a remarkable feature in a HTM based disaster preventionsystem.To evaluate the model’s ability of prediction we calculated the error and average it forevery time window (1200 time step points). The graph in figure 4 on the top shows thisvalue over a span of over one million time steps (horizontal axis are averaged steps, eachof them is 1200 simulation time-steps). The bottom panel of this figure represents theanomaly score averaged on 1200 points, and a characteristic rise and fall of this value isnoticeable. This graph represents the learning curve of the cortical algorithm. The firststeep drop denotes the adaptation to the sensor background noise. After about 12.000time steps the anomaly average is reduced of about 50% the initial value. However, ittakes about 250.000 time steps before the adaptation is complete. After full adaptation,anomaly stays stably to zero, when no seismic waveforms are in input. Subsequently theaverage anomaly seems to drop, but gets much more volatile, indicating the strongerinfluence of the random seismic waveform (see figure 4 bottom panel after t ≈ Figure 4.
In the top panel each points represents the RMS average value for 1200 pointsof simulation. The value is simply the average difference between the predicted value andthe actual acceleration value for 1200 points (so total simulation was running for about600x1200 time steps). The average error seems to remain stable all along the simulation.The bottom panel shows the averaged anomaly score calculated in the same fashion.The value is dropping at the very beginning, then rising again and the decreasing withhigh volatility. This particular shape of the average anomaly was reproduced in differentexperiments with different random seeds.
We have evaluated for the first time how a cortical HTM algorithm can be used torecognize anomalies in seismic signals. We adopted the recent NuPic HTM model [5, 7]and tested its performances on a simulated earthquake prediction experiment. In oursetup the HTM model is fed continuously data from a seismic device. For the most of6/7he time, these data are instrument background noises, however at a defined probability,a seismic waveform is added to the noise and we want to evaluate the HTM model abilityto distinguish this from the background. We found that this system adapt very quicklyto the random fluctuations. After an initial transition time of about 200 thousands timesteps, the HTM cortical algorithm was able to consider the seismic sensor backgroundnoise as normality , lowering its anomaly score to nearly zero. Waveforms instead wererecognized reliably with repeated spikes of high anomaly score.Our tests indicate that the HTM system it is robust to noise, and able to recognizeefficiently small anomalies hidden in the signal. Our study is still qualitative and moreinvestigations are necessary to characterize fully the performance of the algorithm as afeasible earthquake detector. However, on the basis of these results, we feel to speculatethat an HTM setup can help to the realization of robust earthquake detection algorithmsand contribute to intelligent anti-disaster programs.
Acknowledgments
We thank Kahoko Takahashi for the help with citations and for reading this manuscript.
References
1. A. Rex. Automatic earthquake recognition and timing from single traces.
Bulletinof the Seismological Society of America , 68(5):1521–1532, Oct. 1978.2. H. Dai and C. MacBeth. Automatic picking of seismic arrivals in local earthquakedata using an artificial neural network.
Geophys. Jou. Int. , 120:758–774, 1995.3. J. Akram and D. W. Eaton. A review and appraisal of arrival-time picking methodsfor downhole microseismic data.
Geophysics , 81(2):KS71–KS91, MAR-APR 2016.4. H. Adeli and A. Panakkat. A probabilistic neural network for earthquake magnitudeprediction.
Neural Networks , 22(7):1018–1024, SEP 2009.5. Y. Cui, S. Ahmad, and J. Hawkins. Continuous online sequence learning with anunsupervised neural network model.