Neutron-Induced, Single-Event Effects on Neuromorphic Event-based Vision Sensor: A First Step Towards Space Applications
Seth Roffe, Himanshu Akolkar, Alan D. George, Bernabé Linares-barranco, Ryad Benosman
DDate of publication xxxx 00, 0000, date of current version xxxx 00, 0000.
Digital Object Identifier 10.1109/ACCESS.2017.DOI
Neutron-Induced, Single-Event Effectson Neuromorphic Event-based VisionSensor: A First Step Towards SpaceApplications
SETH ROFFE , HIMANSHU AKOLKAR , ALAN D. GEORGE , BERNABÉLINARES-BARRANCO , AND RYAD BENOSMAN.
2, 3, 4 University of Pittsburgh, 4420 Bayard St. Suite 560, Pittsburgh, PA 15213 (emails: {seth.roffe,alan.george}@pitt.edu) University of Pittsburgh, Biomedical Science Tower 3, Fifth Avenue, Pittsburgh, PA 15260 (emails: {akolkar,benosman}@pitt.edu) INSERM UMRI S 968; Sorbonne Université, UPMC Univ. Paris 06, UMRS 968; CNRS, UMR 7210, Institut de la Vision, F-75012, Paris, France Carnegie Mellon University , Robotics Institute, 5000 Forbes Avenue Pittsburgh PA 15213-3890, USA Instituto de Microelectrónica de Sevilla, CSIC and Universidad de Sevilla, Sevilla, Spain (email: [email protected])
Corresponding authors: Seth Roffe (e-mail: [email protected]), Ryad B. Benosman (e-mail: [email protected]).This research was supported by SHREC industry and agency members and by the IUCRC Program of the National Science Foundationunder Grant No. CNS-1738783. This work was performed, in part, at the Los Alamos Neutron Science Center (LANSCE), a NNSA UserFacility operated for the U.S. Department of Energy (DOE) by Los Alamos National Laboratory (Contract 89233218CNA000001).
ABSTRACT
In this paper, we study the suitability of neuromorphic event-based vision cameras for spaceflight, andthe effects of neutron radiation on their performance. Neuromorphic event-based vision cameras are novelsensors that implement asynchronous, clockless data acquisition, providing information about the changein illuminance greater than ( ≥ dB ) with sub-millisecond temporal precision. These sensors have hugepotential for space applications as they provide an extremely sparse representation of visual dynamics whileremoving redundant information, thereby conforming to low-resource requirements. An event-based sensorwas irradiated under wide-spectrum neutrons at Los Alamos Neutron Science Center and its effects wereclassified. Radiation-induced damage of the sensor under wide-spectrum neutrons was tested, as was theradiative effect on the signal-to-noise ratio of the output at different angles of incidence from the beamsource. We found that the sensor had very fast recovery during radiation, showing high correlation of noiseevent bursts with respect to source macro-pulses. No statistically significant differences were observedbetween the number of events induced at different angles of incidence but significant differences werefound in the spatial structure of noise events at different angles. The results show that event-based camerasare capable of functioning in a space-like, radiative environment with a signal-to-noise ratio of 3.355. Theyalso show that radiation-induced noise does not affect event-level computation. Finally, we introduce theEvent-based Radiation-Induced Noise Simulation Environment (Event-RINSE), a simulation environmentbased on the noise-modelling we conducted and capable of injecting the effects of radiation-induced noisefrom the collected data to any stream of events in order to ensure that developed code can operate in aradiative environment. To the best of our knowledge, this is the first time such analysis of neutron-inducednoise analysis has been performed on a neuromorphic vision sensor, and this study shows the advantage ofusing such sensors for space applications. I. INTRODUCTION
Neuromorphic event-based cameras are remarkably efficient,robust, and capable of operating over a large range of lightintensities. These sensors replicate the design of biological retinas to make full use of their power efficiencies, sparseoutput, large dynamic range, real-time computation, and low-data bandwidth. Neuromorphic sensors are built by copyingaspects of their biological counterparts, and are therefore a r X i v : . [ phy s i c s . i n s - d e t ] J a n assively parallel and highly non-redundant [1]. Each pixelof the sensor works independently, sensing changes in lightand providing output in the form of discrete events signifyingincreasing or decreasing light intensity.Event-based cameras are a perfectly suited to space mis-sions where the resource budget is limited and radiation canhave catastrophic effects on hardware. These sensors have thepotential to improve numerous space applications, includingthose involved in space situational awareness, target tracking,observation and astronomical data collection [2]. Due to theharsh conditions entailed, however, the performance of suchsensors in space is yet to be explored. The scope of this workis to test the resilience of neuromorphic sensors to neutronsimpacting the sensor in a highly radiative environment. Thegoal is to determine the failure modes of the neuromorphiccamera as seen under the same spectrum as that produced bycosmic rays and to measure the possible impact of neutronson the temporal precision of output events, noise levels, andcomputation.Although studies have been carried out into the behavior ofvarious optoelectronic devices under neutron radiation [3] [4][5] [6] [7] [8], no work to date has addressed the radiation-tolerance aspects of event-based visual sensors to analyzeif this technology is capable of retaining its efficacy underradiative conditions. To observe and evaluate single-eventeffects, we irradiated a neuromorphic event-based sensor atLos Alamos National Lab’s (LANL) ICE-II neutron facility.The measured neutron energy distribution at LANL-ICE-II is significantly more intense than the flux of cosmic-ray-induced neutrons, and this allows for testing at greatly accel-erated rates. An ICE-II radiation test of less than an hour isequivalent to many years of neutron exposure due to cosmic-rays [9]. Neutrons are known to interact with the materialsin the semiconductor and produce daughter particles, whichmay deposit or remove charge in sensitive volumes of thechip. If the deposited charge is significant enough, it canchange the state of a bit in the system. In a digital system thischange of state is known as a bit-flip. Sensors include analogcircuitry, and therefore produce more complex behavior thansimple bit-flips. Beam testing is popular in sensor processingto classify single-event effects (SEEs) in new computingsystems and test the robustness of systems to single-eventupsets (SEUs). Different systems may respond in differentways to the radiation that brings about SEEs, producingfaults and errors of varying degrees. The affect of SEEs canrange from negligible, where an unused area of memoryis affected, to single-event latch-ups that could damage thesystem permanently.Knowing how a system may respond to radiation is vitalto the success of a space mission insofar that it providesan overview of the kind of upsets that may arise. Thisinformation allows designers to plan for any problems thatmay be encountered in flight. Single-event upsets (SEUs) aretransient in that they do not permanently damage the device,but they may cause some silent data or control errors which,if uncaught, may lead to a loss of performance or accuracy. To reduce risk, it is therefore vital to know how a new systemwill respond to radiation before deployment.In this paper, we measured the effect of radiation andcategorized the SEEs observed in the sensor. We also testedhow radiation affects pure event-based computation in thecontext of optical flow estimation, which is known to besensitive to noise and temporal imprecision, under both ra-diation and non-radiation conditions. Finally, we also usedthis preliminary data to develop a simulator that makes itpossible to inject events with radiation-noise effects intoany data stream. We call this simulator the "Event-basedRadiation-Induced Noise Simulation Environment," (Event-RINSE). Event-RINSE allows realistic neutron beaming ef-fects to be added to any event based data sequence. Thesesimulated radiation effects enable designers to test developedalgorithms prior to mission deployment. II. BACKGROUND
This section gives an overview of the neuromorphic event-driven visual sensor, its data acquisition principles, and itsdata types. The use of event-driven sensors for space appli-cations is also discussed.
A. NEUROMORPHIC EVENT-DRIVEN VISUAL SENSORS
Biomimetic, event-based cameras [10] are a novel type of vi-sion sensors that, like their biological counterparts, are madeof independent cells/pixels which are driven by events takingplace in their field of view, generating an asynchronousstream of spikes/events. This method of data collection is incontrast to conventional vision sensors which are driven byartificially created timing and control signals (frame clock)to create full images that have no relation to either thecontent or the temporal dynamics of the visual scene. Overthe past few years, several types of these event-based camerashave been designed. These include temporal contrast visionsensors sensitive to change in relative luminance, gradient-based sensors sensitive to static edges, devices sensitive toedge-orientation, and optical-flow sensors.Most of these vision sensors output visual informationabout the scene in the form of discrete events using Address-Event Representation (AER) [11] [12] [13]. The data encodesthe visual information by sending out tuples [ x ; y ; t ; p ] —of space (the pixel where change occurred), time (when thechange occurred), and polarity (whether luminance increasedor decreased) — as ON or OFF events, respectively. Theevent-based camera used in this work is a time-domain en-coding event-based sensor with VGA resolution. The sensorcontains a 640 ×
480 array of fully autonomous pixels, eachrelying on an illuminance-change detector circuit. In thisstudy, we will only consider the luminance change circuit thatis common to all existing event-based sensors [14].The operating principle of an event-based pixel is shown inFigure 1. The change detector of each pixel individually de-tects a change in brightness in the field-of-view. Since event-based cameras are not clocked like conventional cameras,2IGURE 1: Event-based sensor operating principles: (A) The event-based sensor used in this experiment. (B) When a givenpixel’s luminosity change reaches a given threshold, it produces a visual event with an x and y address, a timestamp, and apolarity, which is either ON or OFF depending on the change in relative luminosity. (C,D) The stream of events generated bythree rotating shapes, shown here in a color version of the sensor’s absolute light measurement output that comes with everyevent.the timing of events can be conveyed with a very accuratetemporal resolution in the order of microseconds and below .These sensors capture information predominantly in thetime domain as opposed to conventional frame-based cam-eras, which currently provide greater amount of spatial in-formation. Since the pixels only detect temporal changes,redundant information like static background is not capturedor communicated, resulting in a sparse representation of thescene. Consequently, event-based cameras can have a hightemporal-resolution with a very low data-rate [16] comparedto conventional cameras, thus conforming to low-resourcerequirements. Since the pixels are independent of one anotherand do not need a clock, an error in a few of them will notlead to a catastrophic failure of the device and the sensor willbe able to remain operational. B. CONVENTIONAL SPACE SITUATIONAL AWARENESS
Space situational awareness (SSA) has been an importanttopic in military applications for many years [2] [17] [18][19]. SSA is the ability to detect and keep track of surround-ing objects and debris to avoid collisions. For SSA, visionsystems with high temporal-resolution and low latency arerequired to accurately detect objects. Event-based camerasare therefore the perfect candidate to replace limited conven-tional sensing methods in satellite awareness.Ender et al. [20] details the use of radar in SSA for colli-sion detection, orbit estimation, and propagation. The benefitof radar is that it has a very large coverage, meaning it canconsistently observe a wide area in an arc of almost 5000 km.However, since radio uses long wavelengths, this methodol-ogy would only work for larger objects [20]. Smaller objectswould be impossible to detect via radio waves.One difficulty in object detection to avoid collisions inspace is the modeling of non-linear orbits in real-time. Sev-eral methods have been proposed to predict non-linear orbits The highest reported neuromorphic sensor event output rate to date is . × events per second [15]. for SSA. One is to use Gaussian mixture modeling to exploitproperties of linear systems to extrapolate information abouta non-linear system, and then to use Gaussian splitting toreduce the errors induced by that extrapolation [21]. The mix-ture model enables complex, non-linear orbits to be mappedmore accurately, providing a better judgment of potentialcollisions. The issue arises when this kind of surveillancefor object avoidance needs to be done autonomously. Thecalculations presented are too complex to be performed ef-ficiently by a satellite’s embedded platform. Also, since theanalysis carried out by such platforms is based on statisticalmanipulation, it needs to be verified by human interventionin order to avoid any statistical anomalies that may causepotential collisions.Abbot and Wallace [22] tackle the SSA problem of de-cision support for tracking large amounts of orbiting spacedebris. They claim that the limited number of sensors leadsto inconsistent surveillance of the objects under observa-tion, and therefore propose a cooperative monitoring algo-rithm for geosynchronous earth orbit satellites to addresscollision prevention and provide automated alerts. However,this methodology relies on Bayesian modeling, which canbe computationally intensive for embedded platforms andrequires publicly available data to create the models. Withsatellites of unknown orbits, unexpected collisions couldtherefore become an issue.These techniques also require fast positional capture of theobserved objects which is difficult with the video camerascurrently available for space exploration. Event-based cam-eras could fill this space by providing low latency/resourcessensing for SSA. C. EVENT BASED SENSORS FOR SPACE SITUATIONALAWARENESS
The high dynamic range of event-based sensors with bothlow-light and bright-light sources allows visual informationto be inferred even in the darkness of space or when a brightsun is in the sensor’s field-of-view (FoV). It also means that3he area around the sun can be observed, even when the sunis coming up over the horizon of a satellite’s orbit.The use of event-based cameras in space-related appli-cations is not well developed. Most of the work has beencarried out in the context of terrestrial telescope observationof low brightness objects in Low-Earth Orbit (LEO) andGeostationary-Earth Orbit (GEO) [23] [24].Event-based cameras can offer a promising solution tocollision avoidance in space provided their high temporalprecision and sparsity of data are properly taken into accountwhen designing algorithms. The current trend of generatingframes of events, and gray levels to recycle decades of con-ventional computer vision and machine learning techniqueshas led to their being used as simple high dynamic rangeconventional cameras. In this work we focus only on thetemporal properties of these sensors, considering cases ofper-event computation that preserve the temporal propertiesof event-based cameras that have been shown to be the key todeveloping new applications [25].There has been extensive research into event-based cam-eras for real-time tracking and low-power computer systemswithin the last decade. Many algorithms have been developedthat allow for objects to be tracked within the visual space ofan event-driven sensor. Reverter et al. developed one suchmethod that makes it possible to track many different shapes,as long as the pattern of the shapes is known a-priori [26].Similarly, Lagorce et al. provide a multi-kernel Gaussianmixture model tracker for the detection and tracking of differ-ent shaped objects [27]. Other methods use spatial matchingto allow object tracking even in occluded conditions [28][29] and provide haptic stability by tracking gripper positionsin microrobotics applications [30]. The low computationalrequirements of event-based sensors even allow trackingsystems to be implemented on embedded platforms [31] andon FPGAs [32]. Newer improved spatio-temporal featuredetection could improve these methods further [33]. Novelmethods can even detect and track objects in conditionswhere both the camera and the objects are moving indepen-dently [30], [34] [35].
D. NEUTRON-BEAM TESTING
Srour and McGarrity [36] detail the effects of space radiationon microelectronic circuits, discussing damage, ionization,and SEEs on optoelectronic devices. Modern models de-scribe the most of the radiation experienced in the space envi-ronment as consisting of protons and heavy ions [37]. How-ever, this experiment primarily uses wide-spectrum neutronsto test the sensor of interest. In general, neutron beam testingis useful for classifying single-event effects in electronics.Since interest is focused on the response of the device, thesource of the upsets become irrelevant. Neutron testing isalso useful to test the robustness of systems to SEUs. Asan example, NASA Langley Research Center and Honeywellperformed neutron beam tests to study the robustness of theirflight control computer architecture [38]. Their primary goalwas to show that they were able to recover from neutron- induced SEUs. The recovery demonstrated system’s capabil-ities in a hazardous environment, even though the radiationsource was not neutrons.When radiation impacts a device, energy is deposited intothe target material, causing various faults in the hardware.These faults can have different effects such as memory cor-ruption or glitches in analog and digital hardware [39]. Inan imaging sensor, these errors would manifest as corruptedpixels or improper output. One type of effect, single-eventeffects (SEEs), occurs when a high-energy particle strikes amicroelectronic component and changes a single state of theinternals in the device [36]. These effects can then manifestas transient-data errors, corrupting the data output.
III. METHODOLOGY
This section gives an overview of how the radiation exper-iment was performed, explaining the Los Alamos NeutronScience Center’s neutron beam and detailing how data wascollected during irradiation.
Neutronbeam source Event basedsensor Facingbeam90°frombeam (A)(B)
Event basedsensorNeutronbeam source
FIGURE 2: (A) The event-driven sensor under test sitting ona stand that is non-reactive to neutron radiation. To ensurethat the neutrons passed through the sensor, the green laserwas used to aim the beam. (B) Schematics showing thesensor placed at a fixed distance from the beam source intwo conditions - facing the beam directly and at a 90 ° angleof incidence.
A. EVENT-CAMERA
The sensor used for the experiments in this paper wasan event-based sensor based on [14] with VGA resolution(640 ×
480 pixels) fabricated in 180nm CMOS-CIS technol-ogy. The chip has a total die size of 9.6 × , with a pixelsize of 15 × µm , and a fill factor (ratio of photo-diode4rea over total pixel area) of 25%. The maximum event-ratefor this camera is specified as 66 Meps (mega events persecond). During recordings, output events were time-stampedwith micro-second resolution by the camera interface andcommunicated via USB to a host computer for storage. Inour recordings we observed a maximum of about 30 eventscaptured with the same micro-second timestamp, meaningthat the maximum sensor throughput was not reached. B. IRRADIATION
The event-camera under test was irradiated at ICE-II, LosAlamos Neutron Science Center’s wide-spectrum neutron-beam facility. The Los Alamos Neutron Science Center(LANSCE) provides the scientific community with intensesources of neutrons, which can be used to perform experi-ments supporting civilian and national security research. TheICE facility was built to perform accelerated neutron testingof semiconductor devices. Flight Path 30L and 30R, knownas ICE House and ICE-II, allow users to efficiently set up andconduct measurements [9]. The sensor was irradiated for twodays, from November 23, 2019 to November 24, 2019 underwide-spectrum neutrons of energies ranging from . M eV to > M eV . The general setup is shown in Figure 2.An event-based camera was placed at a fixed distance inthe beam to act as a control on the effective neutron flux. Thesensor was placed at different angles of incidence from thebeam as shown in Fig. 2(B) to detect any potential differencesin the effects observed. Data was collected at an angle of ° from the beam and directly facing the beam source.In this experiment, the event-camera was irradiated withthe lens cap on to avoid any light or environmental noiseon the sensor. Thus, the noise recorded from the sensorin this experiment primarily come from the effects of theradiation rather than those induced by the light sources inthe environment. C. DATA COLLECTION AND ANALYSIS
The sensor was connected to a computer running softwarewhich interfaced with the sensor to record events. Eventswere later processed offline. Data was taken with the beamon and off in order to observe the increase in noise caused byirradiation. Radiation-induced noise can be seen in the formof clustered noise-like patterns and line streaks of movingparticles in the focal plane, as will be detailed in the followingsections. The recorded data was parsed to get an event rate tomeasure the number of events generated by the sensor persecond. The counted events were then separated into ON andOFF events. The average events per second were calculatedfor each experiment with standard deviation as error.Data was collected with the sensor facing the beam sourceand at ° , to observe how the angle of incidence affected theincoming radiation noise. The number of events was mea-sured for both ON and OFF events in each orientation andcompared. A Mann-Whitney U test was used to determinestatistical significance in the differences between the twoorientation distributions [40]. FIGURE 3: Average number of noise events per second in-duced due to radiation compared to noise without irradiationover 2 days of irradiation. The recordings were taken with thelens cap on the camera, so the induced events were due eitherto the inherent thermal noise or to noise induced throughthe neutrons. Radiation induced more ON events than OFFevents (3:1 ratio). (A)(B) FIGURE 4: Probability density of events by location onthe sensor for (A) 0 degree angle of incidence and (B) 90degree angle of incidence. The graphs show that the entiresensor was radiated uniformly over the field of view for bothconditions.This experiment measured patterns influenced by the ef-fective neutron flux and the number of ON events and OFFevents. The patterns were analyzed using an understandingof the sensor’s internal circuitry to determine the physicaleffect of radiation on the sensor. This methodology presents5 categorization of SEEs in the form of radiation-inducednoise.To ensure the radiation-induced noise would not over-whelm signal integrity, a pendulum was placed in the visualfield to measure the signal-to-noise ratio. Since the signalcould be observed with and without radiation-induced noise,the signal-to-noise ratio could be calculated by simply divid-ing the number of signal events by the noise events producedby radiation. This ratio could then be used to determine therobustness of the sensor to radiation in terms of loss of signalintegrity. To validate the signal-to-noise ratio, a correlationtest was performed between the radiated data and the non-radiated data.
IV. RESULTS
This section gives an overview of the results of the radiationexperiment, discussing noise rates, patterns, and analyses.
A. INDUCED-EVENT RATE
Data was collected with the lens cap on the sensor to min-imize environmental influence from external lighting. First,the mean number of radiation-induced ON and OFF eventsper second was measured. The average number of events canbe seen in Figure 3. A significant bias towards ON events wasobserved.The induced-event probability density was plotted againstthe pixel coordinates of the sensor to observe any locationpreferences for upsets. To measure this, the pixel location ofeach induced event was divided by the total number of eventsmeasured for both angles of incidence. These measurementscan be seen in Figures 4(A) and 4(B).In both cases, the induced events are quite uniform acrossthe sensor, with the ° angle of incidence tending to biastowards the location of the neutron beam’s 1 inch diameter.We can see that about twice as many events were producedfor high x and low y values than for the opposite corner.However, this is due to human error in placing the sensor inthe beam path. In other words, there is no particular area ofthe sensor that is more vulnerable to neutron radiation effectsthan other areas. This is further demonstrated in the ° angleof incidence result. Every pixel across the sensor showed asimilar response. B. ANGLE OF INCIDENCE COMPARISON
Data was collected at two orientations: facing the beam withan angle of incidence of ° and at an angle of incidence of ° from the beam source. These two distributions were thenanalyzed separately to observe any significant differences.Figure 5 shows that there was a slight difference betweenthe number of OFF events per second induced between thetwo orientations. A Mann-Whitney U test was performed onthe two distributions to test for statistical significance [40]but no statistically significant difference was found. FIGURE 5: Events observed at different angles of incidence.Data was collected at ° from the beam and facing di-rectly towards the beam. No significant difference was foundbetween the number of noise events generated for the twoconditions even though the sensor would be expected tointeract with more neutrons when facing the beam. At °,more events were produced at high x and low y values thanfor the opposite corner. This is the result of human error inplacing the sensor in the beam path. As expected, every pixelacross the sensor showed a similar response.FIGURE 6: Number of events induced in a × pixelbounded box for a light room vs a dark room. Given thecontrast sensitive nature of the sensor, and as expected, weobserved that more ON noise events were generated in thecase of dark room since the neutron interactions allowed forthe event generation threshold to be crossed more often. TheOFF noise events did not increase significantly. C. EFFECTS OF ROOM BRIGHTNESS
When deployed in space, these vision sensors may be subjectto varying levels of background light intensity. To understandhow neutron radiation would affect the sensor under suchvarying conditions, we recorded background noise eventsduring radiation while placing the sensor in an artificially litroom with illuminance levels of around 500 lux and with alens cap covering the sensor to simulate a low-light intensitycondition with a light level close to 0 lux . The intrinsiccharacteristics of the sensor pixels allow them to be invariantto the background lighting conditions thanks to the relativechange operation mode and the log scale. Figure 6 shows thenumber of ON and OFF events induced by neutron radiationin the artificial lit "light room" and in the low-intensity "darkroom" case. We find that the number of ON events induced6IGURE 7: Number of signal events observed vs radiation-induced noise events. Signal events were calculated as therate of events while recording a cyclic pendulum where asthe noise rate was computed from isolated radiation inducedevents. The signal-to-noise ratio for the sensor even understrong neutron radiation was found to be 3.355.in the dark room was nearly . times higher than in the lightroom. Conversely, no significant difference were observed inthe OFF events induced in the two conditions. Details of thisprocess are explained in Section V. D. SIGNAL-TO-NOISE
In order to measure the signal-to-noise ratio, events werecompared with the beam ON and OFF while the sensorobserved a dynamic scene composed of a cyclic-pendulum,as shown in Figure 8(A). To calculate the ratio between thetwo values, the number of signal events measured with thecyclic-pendulum were compared directly with the number ofisolated radiation-induced noise events. This comparison canbe seen in Figure 7. Comparing these values gives a signal-to-noise ratio of . .To ensure the signal can be seen even when radiation isintroduced, events in a × -pixels bounding box (shownin green and red in Figure 8(A)) were measured and plottedto compare signal data with and without radiation. Sincethe pendulum’s movement is cyclic, we calculated the eventrates data over time using a moving window of 1 ms. Thefrequency of this rate data calculated using Fourier transformshould ideally give us the frequency of oscillation of thependulum. The Fourier transform of the signal with andwithout radiation is shown in Figure 8(B). With the additionof radiation noise, the signal’s major frequency can still beestimated with some slight noise at low frequencies.To validate the signal-noise ratio of the radiated sensor, aPearson-correlation test was performed between the radiationdata and the non-radiation data. With a high correlation, it canbe shown that the two distributions follow each other closelywith minor linear transformations. Due to the varying size ofsamples, sub-samples were taken and analyzed to estimatethe correlation R-value. The distribution of R-values can beseen in Figure 9. The measured R-value was . ± . witha negligible p-value. It can therefore be deduced with highconfidence that the radiation-induced noise is not enough to significantly change the data output from the original, non-radiated data.The ultimate goal of deploying sensors on missions is toobtain useful information from them while in space. Oneof the most fundamental, low-level features that can be ex-tracted from the event stream is motion flow. The optical flowprovides the speed and direction of an object’s movementin the camera plane, where its precision is related to thetemporal properties of events. We computed optical flow onevents captured from the sensor recording the moving pendu-lum system using the aperture-robust event-per-event opticalflow technique introduced in [41]. The average direction ofmovement of one arm of the pendulum inside a boundingbox (shown in Figure 10(A)) is plotted in Figure 10(B).The average angle values follow the expected wave as thearm of the pendulum moves up and down vertically. ThePearson correlation between the two conditions was foundto be 0.7189, showing that movement computation is notaffected by radiation. E. NOISE PATTERNS
Radiation-induced noise, as shown in Figure 11 for both ori-entations, can be categorized into two main groups: clustersand line segments. Line segments represent a line of eventsthat appear across the frame due to a neutron impacting thesensor at a non-zero angle of incidence. Clusters representa random burst of events in a small area. The angle ofincidence between the sensor and the radiation source affectsthe number of line segments. About 5-7 times more linesegments appear with a ° angle of incidence than with a °angle of incidence. Conversely, about twice as many clustersappear with a ° angle of incidence than with a ° angleof incidence. Significantly longer lengths of line segmentsoccurred at °, where streaks of up to 300 pixel lengths wereobserved, whereas smaller streaks, with maximum lengths of30-50 pixels, were seen at °. An example of differences innoise cluster patterns can be seen in Figure 11. These figureswere obtained by analyzing recordings of events, witha duration about a 30-second, while searching for uncon-nected clusters not exceeding 10-pixel in size. Note that inFigure 11(A) more event density can be seen in the corner ofhigh x and low y values than in the opposite corner. This issimilar to what was observed in Figure 4 due to human errorin placing the sensor in the beam path.Analysis of noise line segments showed a burst of ONevents over a fast time frame, followed by a long relaxation-period of OFF events after a short wait time, as shown inFigure 12. This is due to an influx of positive current in thesensor’s photo-diodes creating a burst of ON events, followedby a relaxation period for the current to return to normal,creating OFF events. The ON events burst over about 600-800 µs and the negative event tail is about 10ms long.Viewing the event rate of the bursts, we see peaks of ONevents followed by a long tail of OFF events. This effect isseen within all noise-types and is shown in Figures 13 (B) and(C). Figure 13(E) shows a zoomed view with finer details.7 requency (Hz) -40000 -40 0 40040000-4000004000080000 -100 -80 -60 -40 -20 0 20 40-4-202468 -100 -80 -60 -40 -20 0 20 40-4-202468 Frequency (Hz)
No RadiationRadiation -40 0 40 time(ms) (B)(A)
No RadiationRadiation
FIGURE 8: (A) An orbital pendulum recorded using the sensor and the event rate calculated as the number of events within a1 ms moving window with (red) and without (blue) radiation turned on within a bounded box, as shown in the image panels.The images show the event frames obtained within the time window at different time points in the recording. Qualitatively,the sensor produced similar images for both conditions. (B) Calculated frequency of the pendulum using the event rates. Thefrequency of the pendulum’s motion could be obtained using the FFT in each case. V a l u e D e n s i t y R-Value
FIGURE 9: A Pearson correlation test was performed forthe events obtained from the pendulum’s movement with andwithout radiation. The high correlation and small standarddeviation show that the signals obtained from the two condi-tions were quantitatively similar.Bursts of 5 peaks separated by a time of 16.75 ms can beseen. Each peak has a duration of about 1.6 ms of positiveevents. Consecutive bursts are separated by 8.25 ms withinthe 5 peaks. Consequently, on average, the five peaks occurevery 33.25 ms + 16.75 ms = 50 ms , which is equivalentto 100Hz peaks. This coincides with the LANSCE neutronsource description [42], where the neutron source emits apulse of neutrons at a rate of about 100 Hz. Each such neutron peak is referred to as a neutron “macro-pulse”. V. CIRCUIT-LEVEL INTERACTION INTERPRETATION
High-energy neutron beams are thought of as ionizing ra-diation, which can instantaneously change the charge of anelectric circuit node within the camera sensor chip. Since anevent-camera can capture internal changes with microsecondresolution, these sensors provide a new way of “seeing" fineinteractions taking place between fast radiation particles andthe electronic chip while it is operating. (A) (B) (C) n n e - n p + e - p + n γ FIGURE 14: Three possible free neutron decays. (A) Theneutron passes through the sensor casing without decaying.(B) The neutron decays into a proton and electron. (C) Theneutron decays into a proton and an electron which emitsgamma radiation.8 ime: 742 msecTime: 410 msec Time: 928 msecTime: 259 msec
Time: 848 msec
With Radiation
Time: 86 msec
No Radiation
200 600 -1120
200 600 -11 A v e r a g e A n g l e ( r a d i a n s ) Time (msec) A v e r a g e A n g l e ( r a d i a n s ) (A) (B) No RadiationRadiation
FIGURE 10: The movement directions of different parts of the pendulum system computed from the recorded event streamswith and without radiation. (A) The colors represent the movement directions of the events as indicated by the color wheel. (B)Graphs showing the computed average movement directions for events occurring in a 5ms moving window within the blackbounding box shown in the images. The Pearson correlation coefficient between two signals was 0.7189 indicating that thedirection computation was not affected by the radiation.FIGURE 11: Clustered patterns of noise obtained by searching for clusters with minimum sizes of 10 pixels in 30-secondrecordings. Time slices of 5 ms were processed consecutively, searching for 10-pixel clusters. All clusters detected during 30seconds are grouped in the plots. Events were recorded for a ° angle of incidence and a ° angle of incidence. Significantlymore line segments can be seen at °. At °, fewer, smaller (average 30 pixels) and more clustered noise patterns were observedthan at °, where longer (up to 300 pixels) and more frequent (up to 7 times) line segments were observed.For the free neutrons passing through the sensor, thereare three main possibilities: the neutron can pass throughas a neutron without decaying, the neutron can decay into aproton and an electron, or the neutron can decay into a protonand an electron which emits a gamma photon due to internalbrehmsstrahlung [43]. A diagram of these three possibilities can be seen in Figure 14. Due to quantum uncertainties andthe inability to distinguish between particles, it is impossibleto distinguish the cases’ impact on the sensor in this experi-ment. Further research must therefore be performed to detailthe exact cause of the induced noise patterns.In digital circuits, high-energy charged particles and radi-9 x [ p i x e l s ] t i m e ( m s ) y [ p i x e l s ] T i m e ( m s ) x [ p i x e l s ] y [ p i x e l s ] T i m e ( m s ) (A) (B) (C) FIGURE 12: Details of 20ms event capture when exposed to neutron beam without visual stimulus. The blue dots representpositive events and the red dots represent negative events. Positive events are mostly concentrated in 600-800 µs time intervalsseparated by about 8 ms intervals in which mostly negative events are recorded. (A) 3D plot (x,y,time) of events capturedduring the 20 ms interval. Small scattered dots/clusters can be observed plus a line segment in the lower right part. (B) Time vsx-coordinate projection of the recorded events. (C) Events corresponding to the line segment in (A) which have been isolatedfor better visibility.ation beams tend to mainly impact memory circuits, wherecharge is stored on tiny parasitic capacitors, producing bit-flips and consequently altering system states and data. Inour sensor, however, we observed consistent sudden positiveevents over many pixels followed by negative event tails, syn-chronously with the macro-pulse neutron emission patternsof LANSCE [42]. The fact that most responsive pixels pro-duce a burst of positive events during each 625 µs LANSCEneutron macro-pulse, rules out the possibility that the sen-sor is suffering bit-flip effects at temporary memory-storingnodes. If this were the case, we would expect to observea random mix of positive and negative events within eachneutron macro-pulse. However, most of the affected pixelsrespond by providing a synchronized burst of positive events.It can thus be inferred that it is the pixels’ photo-diodes thatare responding to the neutron macro-pulses. Photo-diodesdrive a photo-current proportional to incident light intensity.If a high-energy proton or electron crosses the depletionregion of a photo-diode, it will interact, either by attractionor repulsion, with the electrons flowing through it at thatmoment, thus producing a sudden decrease in photo-currentand, consequently, negative events. However, since we ob-served a sudden, very significant increase in photo-current(resulting in positive events), we hypothesize that the scat-tered pixels are sensing sudden radiation at their locations.This would also explain the observation of segments sensedsimultaneously by consecutive pixels. Figure 12 shows onesuch segment in a 20 ms time slice of events, correspondingto three consecutive 625 µs neutron macro-pulses separatedfrom each other by 8.25 ms . Most of the pixel responsesshow small clusters of less than 10-pixels, the exceptionbeing the 190-pixel long segment. Our hypothesis is thatthe sensor is crossed by radiation bursts, most of themperpendicular to the chip plane, but occasionally interactingwith deflected radiations at other angles and producing linesegments. However, all radiation interactions occur preciselyduring the beam’s macro-pulse times. The electronic pixel circuitry of an event-camera chip hasa limited response time in the range of 0.1 ms to 10 ms de-pending on ambient light and bias conditions [14] [44]. TheLANSCE neutron source macro-pulses have a time durationof 625 µs , which is lower than the temporal resolution ofthe event sensor. The macro-pulse radiation impinging onthe destination pixels produces a sudden over-stimulation ofphoto-current, resulting in the sudden generation of a handfulof positive events per pixel during the neutron macro-pulse.After such strong over-stimulation, the pixel circuit relaxesto its steady ambient-light-driven state with a time constantin the range of 10 ms , producing events of negative polarityover time. This behavior of sudden positive stimulation of600-800 µs , where positive events are produced, followedby about 8-10 ms of negative-event relaxation is systemat-ically observed in the recordings. Figure 12(A) shows the 20 ms event capture with scattered noise-like dots/clusters offast positive events (shown in blue), followed by negativeevent tails (shown in red). We hypothesize that each suchdot/cluster corresponds to a neutron crossing the chip. Fig-ure 12(B) shows the events in Figure 12(A), but displayedin their corresponding time vs x-coordinate projection. Wecan clearly see the synchronized sequence of neutron macro-pulse-induced positive events (shown in blue), of 600-800 µs duration, separated by about 8 ms of inter-neutron macro-pulse time where mainly negative relaxation events are pro-duced. The figure also shows a 190-pixel long segment withthe same time profile. The events for this segment are isolatedin Figure 12(C). In this plot there are 2,031 positive eventscollected over about 800 µs , followed by 1,090 negativeevents collected during over about 20 ms .The suddenly induced photo-current hypothesis also ex-plains the observations in Figure 6, where more positiveevents are produced under dark-room conditions than underlight-room conditions. When under light room conditions,the photo-diodes are already driving some current and conse-quently reach their maximum saturation current earlier when10 Time (ms)
Time (ms) E V E N T R A T E ( e v e n t s / m s ) NO RADIATIONRADIATION - 90 Degree RADIATION - PENDULUM STIMULIRADIATION - FACING BEAM(A) (B)(D)(C) E V E N T R A T E ( e v e n t s / m s ) μ s)0 1000 2000 3000 4000 5000 6000 7000 8000 9000 μ s 8350 μ s 8350 μ s 8250 μ s 8300 μ s 8300 μ s 8400 μ s 8300 μ s16750 μ s1600 μ s ON EventsOFF Events E V E N T R A T E ( e v e n t s / m s ) (E) ON EventsOFF Events
FIGURE 13: Noise rates for different conditions with and without neutron radiation. (A) The overall noise without radiation isvery low. (B, C) Radiation noise when the sensor was placed at ° (facing) (B) and at ° (C) to the beam source. In each case,we recorded the bursts of noise most likely due to neutron pulses from beam generation. (D) Similar noise was found in therecording when a circular pendulum was recorded with the camera. The burst noise was superimposed on the low frequencyevents generated by the pendulum motion. (E) Details of the neutron’s macro-pulse sequence can be observed from a zoomed-inplot of the event bursts in (B). Each neutron macro-pulse produced positive event bursts with duration of about 1.6 ms , andwith peaks separated on average by 8.3 ms . Five macro-pulse responses appear, with a duration between the first and the fifthof 33.25 ms , while the time between two 5-macro-pulse trains is 16.75 ms .suddenly impinged by high energy particles, resulting infewer induced positive events. Under dark conditions, thephoto-current can undergo a larger variation, resulting inmore positive events. VI. EVENT-RINSE SIMULATOR
A. SIMULATED NOISE GENERATION
Given a stream of event-camera data as input, the simulatorsteps through each event. For every time step in the data,a noise event is either generated or passed. The probabilityof injection was determined using a Poisson distribution of observing k = 1 event with a variable event rate. Namely, P ( λ ) = λe − λ (1)where λ is the frequency of an event happening per microsec-ond. A starting pixel is randomly chosen uniformly acrossthe resolution of the sensor. The simulator decides whetherinjected noise is in the form of a cluster or a line segmentbased on the angle-of-incidence parameter. Specifically, thechance of injecting a cluster is based on the cosine of the an-gle of incidence with some jitter-error. Thus, the probabilityof the injected noise pattern is given by Eq 2.11 oise Patterns (Simulated Data)Noise Patterns (Real Data) T i m e ( μ s ) X-pixels0 50 100 150 200 250 300 X-pixels0 50 100 150 200 250 300 (A) (B)
FIGURE 15: Examples of X-projections of noise line segments for events recorded from (A) real data and (B) simulated noisedata. All cases show a burst of ON (blue) events with a long OFF (red) tail. As each example line segment was detected, theevents were separated from the recording and then projected on the X-axis. N o r m a li z e d E v e n t R a t e Observed Noise
Simulated Noise
ON Events OFF Events
Time( μ s) FIGURE 16: Average single pixel radiation-induced eventrate model for observed and simulated data. From real datawe observed that neutron interactions induced ON eventbursts of about 1.6 ms within the first 1 ms . These werefollowed by long tails of OFF events lasting up to 10 ms .The simulator was used to induce noise events into the streamof recorded non-noisy data, and the noise characteristics forsingle-event noise were then averaged to create the dashedcurves. The simulator was able to match the real noise modelwithin a margin of acceptable error. P ( Cluster ) = | cos ( θ + ε )) | P ( Line Segment ) = 1 − P ( Cluster ) (2)where θ is the angle of incidence in radians and ε is a smallamount of error. A cluster’s shape is modelled by randomlychosen pixels around the neighborhood of the starting point.A line segment is modelled by a straight line with a randomlychosen angle between 1° and 360°.For each pixel in the shape of the generated event, thenoise pattern is modelled by sampling a time window for ONevents from N (2000 µs, µs ) which represents the lengthof time for the burst of ON events. OFF events are sampled from N (8000 µs, µs ) . More precisely, P ( ON N oise Event ; t ) = e −
12 ( t − tON )2 σON t ∈ [0 , t ON ] t ON ∼ N (2000 , (3)where the burst of ON events is simulated as a Gaussianmodel with the mean as the sampled ON event time window( t ON ) and standard deviation σ ON = 340 µs is used todetermine the probability of generating an event over time t .The wait time between the burst of ON events and the OFF-event relaxation period is sampled from N (100 , . Afterthe wait time ( t W ait ), the current relaxation of OFF eventsis modelled using an exponential with decay parameter, β = t OF F ) as per Eq.(4). P ( OF F N oise Event ; t ) = 1 β e − β t t ∈ [ t ON + t W ait , t
OF F ] t W ait ∼ N (100 , t OF F ∼ N (8000 , (4)The generated events are then added to the data file andsorted by timestamp in ascending order. Finally, the file issaved to be used in testing or evaluation. The algorithm togenerate radiation-induced noise events is detailed in Algo-rithm 1. B. PATTERN VALIDATION
To validate the simulation environment, noise events weregenerated following the pattern described in Algorithm 1and compared with noise events from real data. The noiseevents were plotted against time to compare them with noisefrom observations. Figure 15 shows a sample of visual realnoise events (Figure 15(A)) vs simulated noise events (Figure15(B)). The model used to generate noise was compared tothe average observed single-event noise. The model shownin Figure 16 fits the observed pattern with a error rate forON noise profiles and . error rate for OFF noise profile.12 lgorithm 1 Radiation Induced Noise Simulation Environ-ment (Event-RINSE) for At each time step t do Compute chance of radiation-induced noise usingEq.(1) if Generate Noise Event then Decide if noise is cluster or line using Eq.(2) Choose a random pixel [ x , y ] if CLUSTER NOISE then Randomly sample a set of pixels [ X, Y ] in theneighborhood of [ x , y ] for Each pixel ∈ [ X, Y ] in the cluster do Generate ON Events Using Eq.(3)
Generate OFF Events Using Eq.(4) end for end if if LINE NOISE then
Randomly sample angle of line: θ ∈ [0, 2 π ) Select a set of pixels [ X, Y ] forming a line Lstarting at [ x , y ] with angle θ for Each pixel [ X, Y ] of the line do Generate ON Events Using Eq.(3)
Generate OFF Events Using Eq.(4) end for end if end if
Append noise events to stream end for
Sort events by ascending timestamps
C. SIMULATION ENVIRONMENT USAGE
The Event-RINSE simulation environment is written inPython with many supporting parameter flags that can beused to modify the simulation model. Normal Python dataanalysis modules are needed for the simulator, namely SciPy[45] and NumPy [46], while OpenCV [47] is used to displayvideos of the event data. The simulator is run using Python3environment with runtime flags for campaign customization.Currently available flags and descriptions can be seen inTable 1. The input data file is the only input that is necessaryto run the simulator. Input files are assumed to be plain textfiles in < x > < y > < timestamp ( µs ) > < polarity > format. VII. CONCLUSION
The purpose of this experiment was to irradiate an event-based camera under wide-spectrum neutrons to view andclassify any SEEs that may be observed. The results showthat the main SEU that affects the event-based camera isradiation-induced noise in the form of uniformly-distributedevents across the sensor’s field of view. We found that noiseinduced on single pixels resulted in both ON and OFF eventswith a ratio of 3:1. An average noise event rate was foundto generate peaks with lags in the range of 8-10 ms which corresponded directly with the macro-pulse patterns of theneutron source at LANSCE [42]. This shows that the sensoracted like a naive particle detector, and was only affectedby the radiation over short timescales. OFF events werealso seen to follow the ON-event peaks with exponentially-decaying event-rate profile. These profiles seem to suggestthat the neutrons interact with the photo-diode in individualpixels causing energy dumps leading to large photo-current,inducing the ON events in a short time period of about1.6 ms . The residual relaxation current after the radiationpasses gives rise to the OFF events at much lower rates, butwith a longer duration of up to 10 ms . The radiation didnot cause any permanent, long-term damage to the sensor’sphoto-diodes or the hardware circuitry. This hypothesis wasfurther confirmed when looking at the noise events in brighterand darker background-illumination conditions, where ONevents were significantly higher in the dark environment dueto sensor’s higher contrast sensitivity but OFF events werenot found to change significantly across the two conditions.Focusing on induced noise, experiments were performedto observe correlations with the angle of incidence and theevent rate through the sensor. Surprisingly, the null hypothe-sis that there is no correlation between the number of eventsand the angle of incidence, was supported. With a largerangle of incidence, the cross-sectional area of the sensor issmaller to the beam’s point-of-view, making it less likely tobe hit. When a neutron does impact the sensor, however,it travels across the field leaving a long streak of eventsfollowing its trajectory. When there is a smaller angle ofincidence, the sensor looks larger from the perspective of thebeam. This implies that the sensor will be more likely to behit, but events are shown only in the form of dots as shortlines of neutrons penetrate the sensor. These two effects thuscancel each other out, showing no difference in the inducedevent rate.Comparing the number of events from a pendulum signalwith radiation-induced noise shows a signal-to-noise ratio of3.355. This ratio demonstrates the robustness of the event-based sensor to radiation in that the noise introduced doesnot significantly impact its ability to extract features of thedesired signal. This is further illustrated by the sensor’sability to clearly observe the sinusoidal signal against thenoisy background, and by the results of the optical flowalgorithm implemented on the recorded events, which showno significant deterioration between the flow directions com-puted from the events when the radiation is introduced.The Event-RINSE simulation environment created usingthe recorded noise data can be used to simulate the effectsof radiation on pre-recorded data files. Event-RINSE wasused to inject noise into the event streams recorded withoutradiation and was found to correspond well with the observedprofile. The noise examples generated from the simulatormatched both the average single-event noise model and theaverage noise across the sensor. This fault injector makesit possible to test different neuromorphic-sensor algorithms,such as object tracking, under a noisy radiation environment13ABLE 1: Summary of Event-RINSE runtime options. Command Flag Description Datatype-h/–help Display help message and exit N/A-f/–input-file The input data file path to read from String-o/–output-file Custom output data file path to write to String-aoi/–angle-of-incidence Angle of incidence between the sensor andsimulated beam. Affects prevalence of lines vs.clusters Integer-s/–imgSize The size of the images from the sensor data List of 2 integers-vi/–view-input View the input data file as a video N/A-vo/–view-output View the output data file as a video N/A-i/–inject Perform injections on input file and write tooutput file N/A-d/–delta Time-step to hold in one frame when viewingvideo Float-n/–noise The event rate of noise with standard deviation List of 2 integers without the need for expensive radiation testing, and therebyto assess an algorithm’s viability in space and in any noisesuppression techniques. Future work could look at improvingthe parameters and probability models for more accuratenoise generation.Further development of event-cameras for space shouldinclude research into their efficacy under proton and heavy-ion radiation. These experiments will show if the sensor, asit currently stands, is capable of survival under the harshconditions of space. Future work could also include testingthe sensor’s capability to perform basic object tracking underneutron irradiation. The noise shown in this experiment couldpose a small problem for SSA by interfering with signalevents in object tracking. However, since the noise wasseen to be fairly constant under various cases, it could bemodeled for background analysis. Also, the induced noisedid not appear to deteriorate signal analysis enough to causedetrimental effects. With minor background suppression, thesignal-to-noise ratio could therefore be improved enough toperform the necessary algorithms and analysis for SSA onfuture spacecraft.
ACKNOWLEDGMENTS
This research was supported by SHREC industry and agencymembers and by the IUCRC Program of the National ScienceFoundation under Grant No. CNS-1738783. This work wasperformed, in part, at the Los Alamos Neutron ScienceCenter (LANSCE), a NNSA User Facility operated for theU.S. Department of Energy (DOE) by Los Alamos NationalLaboratory (Contract 89233218CNA000001). We would alsolike to thank M. Lozano, M. Ullán, S. Hidalgo, C. Fleta,and G. Pellegrini from the "Instituto de Microelectrónica deBarcelona" (IMB-CSIC) for insightful discussions.
REFERENCES [1] C. Posch, T. Serrano-Gotarredona, B. Linares-Barranco, and T. Delbruck,“Retinomorphic event-based vision sensors: Bioinspired cameras withspiking output,” Proceedings of the IEEE, vol. 102, pp. 1470–1484, 102014.[2] D. Oltrogge and S. Alfano, “The technical challenges of better spacesituational awareness and space traffic management,” Journal of SpaceSafety Engineering, vol. 6, 05 2019.[3] S. Watts, J. Matheson, I. Hopkins-Bond, A. Holmes-Siedle, A. Moham-madzadeh, and R. Pace, “A new model for generation-recombination insilicon depletion regions after neutron irradiation,” Nuclear Science, IEEETransactions on, vol. 43, pp. 2587 – 2594, 01 1997.[4] M. El Mashade, M. Ashry, S. Eladl, and M. Rageh, “Experimentalmeasurements of some optoelectronic devices before and after gammairradiation,” vol. 3, pp. 1–12, 01 2004.[5] Z. Kovalyuk, V. Katerynchuk, O. Politanska, O. Sydor, and V. Khomyak,“Effect of gamma radiation on the properties of inse photodiodes,” Tech-nical Physics Letters, vol. 31, pp. 359–360, 05 2005.[6] M. Vujisic, K. Stankovic, N. Marjanovic, and P. Osmokrovic, “Simulatedeffects of proton and ion beam irradiation on titanium dioxide memristors,”Nuclear Science, IEEE Transactions on, vol. 57, pp. 1798 – 1804, 09 2010.[7] D. Alexander, “Transient ionizing radiation effects in devices and circuits,”Nuclear Science, IEEE Transactions on, vol. 50, pp. 565 – 582, 07 2003.[8] B. Iricanin and D. Gvozdic, “The analytic approach in the modelling ofone-dimensional electron concentration distribution in some two-valleysemiconductor electron devices,” Journal of Physics A: Mathematical andGeneral, vol. 31, p. 2997, 01 1999.[9] N. Suzanne, W. Stephen, and M. Michael, “The los alamos neutron sciencecenter spallation neutron sources,” Physics Procedia, vol. 90, pp. 374–380,12 2017.[10] P. Lichtsteiner, C. Posch, and T. Delbruck, “A ×
120 dB 15 µ sLatency Asynchronous Temporal Contrast Vision Sensor,” IEEE Journalof Solid-State Circuits, vol. 43, no. 2, pp. 566–576, 2008.[11] M. A. Mahowald, “VLSI analogs of neuronal visual processing: A syn-thesis of form and function,” Ph.D. dissertation, California Institute ofTechnology, 1992.[12] J. Lazzaro and J. Wawrzynek, “A multi-sender asynchrounous extension tothe AER protocol,” in Conference on Advanced Research in VLSI, 1995.[13] K. A. Boahen, “Point-to-point connectivity between neuromorphic chipsusing address-events,” IEEE Transactions on Circuits and Systems, vol.47-5, pp. 416 – 434, 2000.[14] P. Christoph, M. Daniel, and W. Rainer, “A QVGA 143 db dynamic rangeframe-free PWM image sensor with lossless pixel-level video compressionand time-domain CDS,” IEEE Journal of Solid-State Circuits, vol. 46,no. 1, pp. 259–275, 2011.[15] Y. Suh, S. Choi, M. Ito, J. Kim, Y. Lee, J. Seo, H. Jung, D. H. Yeo,S. Namgung, J. Bong, S. Yoo, S. H. Shin, D. Kwon, P. Kang, S. Kim,H. Na, K. Hwang, C. Shin, J. S. Kim, P. K. J. Park, J. Kim, H. Ryu,and Y. Park, “A 1280x960 dynamic vision sensor with a 4.95-um pixelpitch and motion artifact minimization,” in IEEE Int. Symp. Circuits andSystems (ISCAS), 2020, pp. 1–5.[16] R. Benosman, S.-H. Ieng, C. Clercq, C. Bartolozzi, and M. Srinivasan,“Asynchronous frameless event-based optical flow,” Neural Networks,vol. 27, pp. 32–37, 2012.
17] J. Kennewell and B.-N. Vo, “An overview of space situational awareness,”01 2013, pp. 1029–1036.[18] G. Gasparini and V. Miranda, “Space situational awareness: an overview,”01 2010.[19] J. Pelton, “A path forward to better space security: Finding new solutions tospace debris, space situational awareness and space traffic management,”Journal of Space Safety Engineering, vol. 6, 05 2019.[20] E. Joachim, L. Ludger, B. Andreas, and W. Helmut, “Radar techniques forspace situational awareness,” in 2011 12th International Radar Symposium(IRS). IEEE, 2011, pp. 21–26.[21] D. K. Jordan, “Nonlinear orbit uncertainty prediction and rectification forspace situational awareness,” Ph.D. dissertation, The University of Texasat Austin, 2010.[22] I. A. Richard and P. W. Timothy, “Decision support in space situationalawareness,” Lincoln Laboratory Journal, vol. 16, no. 2, p. 297, 2007.[23] N. Ralph, D. Maybour, Y. Bethi, and G. Cohen, “Observations and designof a new neuromorphic event-based all-sky and fixed region imaging sys-tem,” Proceedings of the Advanced Maui Optical and Space SurveillanceTechnologies Conference, AMOS, vol. PP, pp. 17–20, 09 2019.[24] S. Afshar, A. Nicholson, A. van Schaik, and G. Cohen, “Event-based ob-ject detection and tracking for space situational awareness,” IEEE SensorsJournal, vol. PP, pp. 1–1, 07 2020.[25] H. Akolkar, C. Meyer, X. Clady, O. Marre, C. Bartolozzi, S. Panzeri, andR. Benosman, “What can neuromorphic event-driven precise timing add tospike-based pattern recognition?” Neural computation, vol. 27, pp. 561–593, 03 2015.[26] D. R. Valeiras, X. Lagorce, X. Clady, C. Bartolozzi, S. Ieng, and R. Benos-man, “An asynchronous neuromorphic event-driven visual part-basedshape tracking,” IEEE Trans. Neural Netw. Learn. Syst. (TNNLS), vol. 26,pp. 3045–3059, 2015.[27] X. Lagorce, C. Meyer, S. H. Ieng, D. Filliat, and R. Benosman, “Asyn-chronous event-based multikernel algorithm for high-speed visual featurestracking,” IEEE Trans. Neural Netw. Learn. Syst. (TNNLS), vol. 26, pp.1710–1720, 2015.[28] Z. Ni, S. H. Ieng, C. Posch, S. Regnier, and R. Benosman, “Visualtracking using neuromorphic asynchronous event-based cameras,” NeuralComputation, vol. 27, pp. 925–953, 2015.[29] L. A. Camuñas-Mesa, T. Serrano-Gotarredona, S. Ieng, R. Benosman,and B. Linares-Barranco, “Event-driven stereo visual tracking algorithmto solve object occlusion,” IEEE Transactions on Neural Networks andLearning Systems, vol. 29, no. 9, pp. 4223–4237, 2018.[30] Z. Ni, A. Bolopion, J. Agnus, R. Benosman, and S. Regnier, “Asyn-chronous event-based visual shape tracking for stable haptic feedback inmicrorobotics,” IEEE Trans. Robot., vol. 28, pp. 1081–1089, 2012.[31] M. Litzenberger, C. Posch, D. Bauer, A. N. Belbachir, P. Schon, B. Kohn,and H. Garn, “Embedded vision system for real-time object tracking usingan asynchronous transient vision sensor,” IEEE 12th Digital Signal Proc.Workshop and 4th IEEE Signal Proc. Education Workshop, pp. 173–178,2006.[32] A. Linares-Barranco, F. Gómez-Rodríguez, V. Villanueva, L. Longinotti,and T. Delbruck, “A USB3.0 FPGA event-based filtering and trackingframework for dynamic vision sensors,” IEEE Int. Symp. Circuits andSystems (ISCAS), 2015.[33] X. Lagorce, S.-H. Ieng, X. Clady, M. Pfeiffer, and R. Benosman, “Spa-tiotemporal features for asynchronous event-based data,” Front. Neurosci.,vol. 9, 2015.[34] B. Ramesh, S. Zhang, Z.-W. Lee, Z. Gao, G. Orchard, and C. Xiang,“Long-term object tracking with a moving event camera,” British MachineVision Conf. (BMVC), 2018.[35] A. Mitrokhin, C. Fermüller, C. Parameshwara, and Y. Aloimonos, “Event-based moving object detection and tracking,” IEEE/RSJ Int. Conf. Intelli-gent Robots and Systems (IROS), 2018.[36] J. R. Srour and J. M. McGarrity, “Radiation effects on microelectronics inspace,” Proceedings of the IEEE, vol. 76, no. 11, pp. 1443–1469, 1988.[37] G. Badhwar and P. O’Neill, “An improved model of galactic cosmic radi-ation for space exploration missions,” International Journal of RadiationApplications and Instrumentation. Part D. Nuclear Tracks and RadiationMeasurements, vol. 20, no. 3, pp. 403–410, 1992.[38] E. Kenneth, B. C. M, K. Daniel, H. Richard, S. Kent, V. Linda, andF. Matthew, “Closed-loop neutron particle effects testing on a recoverableflight control computer,” in The 23rd Digital Avionics Systems Conference(IEEE Cat. No. 04CH37576), vol. 2. IEEE, 2004, pp. 6–.[39] M. Bagatin and S. Gerardin, Ionizing Radiation Effects in Electronics:From Memories to Imagers. CRC press, 2018. [40] H. B. Mann and D. R. Whitney, “On a test of whether one of two randomvariables is stochastically larger than the other,” Annals of MathematicalStatistics, vol. 18, no. 1, pp. 50–60, 1947.[41] H. Akolkar, S. H. Ieng, and R. Benosman, “Real-time high speed mo-tion prediction using fast aperture-robust event-driven visual flow,” IEEETransactions on Pattern Analysis and Machine Intelligence, vol. 99, no. 1,pp. 1–12, 2020.[42] S. Wender and L. Dominik, “Los alamos high-energy neutron testinghandbook,” SAE Technical Paper, Tech. Rep., 2020.[43] K. Hagiwara, K. Nakamura, K. Hikasa, M. Tanabashi, M. Aguilar-Benitez,C. Amsler, R. Barnett, O. Dahl, D. Groom, R. Miquel et al., “Review ofparticle physics,” Physical Review. D, Particles Fields, vol. 66, no. 1, pp.010 001–010 001, 2002.[44] T. Serrano-Gotarredona and B. Linares-Barranco, “A 128 ×
128 1.5%contrast sensitivity 0.9% FPN 3 µ s latency 4mW asynchronous frame-freedynamic vision sensor using transimpedance preamplifiers,” IEEE Journalof Solid-State Circuits, vol. 48, no. 3, pp. 827–838, 2013.[45] V. Pauli, G. Ralf, O. T. E., H. Matt, R. Tyler, C. David, B. Evgeni, P. Pearu,W. Warren, B. Jonathan, v. S. J., R. A. H., P. Fabian, v. Paul, and S. . .Contributors, “SciPy 1.0: Fundamental algorithms for scientific computingin python,” Nature Methods, vol. 17, pp. 261–272, 2020.[46] E. O. Travis, A guide to NumPy. Trelgol Publishing USA, 2006, vol. 1.[47] G. Bradski, “The OpenCV Library,” Dr. Dobb’s Journal of Software Tools,2000. ETH ROFFE earned a Bachelor of Philosophyin Physics, Astronomy, and Mathematics from theUniversity of Pittsburgh in 2017 and a M.S degreein Electrical and Computer Engineering from theUniversity of Pittsburgh in 2020. He is currentlypursuing a PhD in Electrical and Computer Engi-neering.He has been a member of the NSF Center forSpace, High-performance and Resilient Comput-ing (SHREC) since 2018 performing research inspace computing under the direction of Dr. Alan George. His main researchinterests involve resilience in sensor processing including data reliability anderror classification in novel sensors.
ALAN D. GEORGE is Department Chair, R&HMickle Endowed Chair, and Professor of Electri-cal and Computer Engineering (ECE) at the Uni-versity of Pittsburgh. He is Founder and Directorof the NSF Center for Space, High-performance,and Resilient Computing (SHREC) headquarteredat Pitt. SHREC is an industry/university coopera-tive research center (I/UCRC) featuring some 30academic, industry, and government partners andis considered by many as the leading research cen-ter in its field. Dr. George’s research interests focus upon high-performancearchitectures, applications, networks, services, systems, and missions forreconfigurable, parallel, distributed, and dependable computing, from space-craft to supercomputers. He is a Fellow of the IEEE for contributions inreconfigurable and high-performance computing.
HIMANSHU AKOLKAR is currently a Post Doc-toral Associate at the University of Pittsburgh.He received his M.Tech. degree from IIT, Kanpur(India) in EE and PhD from IIT, Genoa (Italy) inRobotics after which he had a Post Doctoral stintat Universite Pierre et Marie Curie. His primaryinterest is to understand the neural basis of sen-sory and motor control to develop an intelligentmachine.
BERNABÉ LINARES-BARRANCO
RYAD BENOSMAN received the M.Sc. andPh.D. degrees in applied mathematics and roboticsfrom University Pierre and Marie Curie in 1994and 1999, respectively. He is Full Professorat the university of Pittsburgh/Carnegie Mel-lon/Sorbonne University. His work pioneered thefield of event based vision. He is the cofounder ofseveral neuromorphic related companies includingProphesee and Pixium Vision a french prostheticscompany. Ryad Benosman has authored more than60 publications that are considered foundational to the field of event basedvision and holds several patents in the area of vision, robotics and imagesensing. In 2013, he was awarded with the national best French scientificpaper by the publication LaRecherche for his work on neuromorphic retinasapplied to retina prosthetics.received the M.Sc. andPh.D. degrees in applied mathematics and roboticsfrom University Pierre and Marie Curie in 1994and 1999, respectively. He is Full Professorat the university of Pittsburgh/Carnegie Mel-lon/Sorbonne University. His work pioneered thefield of event based vision. He is the cofounder ofseveral neuromorphic related companies includingProphesee and Pixium Vision a french prostheticscompany. Ryad Benosman has authored more than60 publications that are considered foundational to the field of event basedvision and holds several patents in the area of vision, robotics and imagesensing. In 2013, he was awarded with the national best French scientificpaper by the publication LaRecherche for his work on neuromorphic retinasapplied to retina prosthetics.