beamModelTester: software framework for testing radio telescope beams
AAstronomy and Computing 00 (2019) 1–10
AstronomyandComputing © http://creativecommons.org/licenses/by-nc-nd/4.0/ beamModelTester: software framework for testing radio telescope beams Ois´ın Creaner a, ∗ , Tobia D. Carozzi b a Dublin Institute for Advanced Studies, 31 Fitzwilliam Place, Dublin 2, Ireland b Department of Space, Earth and Environment, Chalmers University of Technology, Onsala Space Observatory, 439 92 Onsala, Sweden
Abstract
The flux, polarimetric and spectral response of phased array radio telescopes with no moving parts such as LOFAR is known to vary considerablywith orientation of the source to the receivers. Calibration models exist for this dependency such as those that are used in the LOFAR pipeline.Presented here is a system for comparing the predicted outputs from any given model with the results of an observation. In this paper, a sampleobservation of a bright source, Cassiopeia A, is used to demonstrate the software in operation, by providing an observation and a model of thatobservation which can be compared with one another. The package presented here is flexible to allow it to be used with other models and sources.The system operates by first calculating the predictions of the model and the results of an observation of linear fluxes and Stokes parametersseparately. The model and observed values are then joined using the variables common to both, time and frequency. Normalisation and RFIexcision are carried out and the di ff erences between the prediction and the observation are calculated. A wide selection of 2-, 3- and 4-dimensionalplots are generated to illustrate the dependence of the model and the observation as well as the di ff erence between them on independent parameterstime, frequency, altitude and azimuth. Thus, beamModelTester provides a framework by which it is possible to calibrate and propose refinementsto models and to compare models with one another. Keywords:
LOFAR, Beam Modelling, Radio Flux, Radio Polarimetry
1. Introduction
This paper presents a software package, beamModelTester (Creaner, 2018–), which is designed to enable the evaluationand comparison of models of the beams of radio telescopes.This is especially valuable for flux calibration of radio tele-scopes with no moving parts such as the LOw Frequency ARray(LOFAR). The antennas in such telescopes are fixed in posi-tion and receive signals from all directions at once, in contrastto traditional, mechanically-slewed radio telescopes where thedish(es) and antenna(s) are moved to point at the target source.Instead, the signals from each of the stationary antennas arecombined using software and electronic components (Butcher,2004). LOFAR has, since its development, been referred toas a “software telescope” (Butcher, 2004). Software is neededto perform all operations on the telescope, and software-basedmodels are used extensively to calibrate the outputs (Butcher,2004). ∗ Corresponding author
Email addresses: [email protected] (Ois´ın Creaner), [email protected] (Tobia D. Carozzi)
In Figure 1, a schematic of the configuration of a singleLow Band Antenna (LBA) element can be seen as an exampleof such an instrument. The element consists of two antennaswhich are aligned orthogonally to one another on axes labelled x and y . These two axes are orthogonal to the z -axis which,in the case of LOFAR, is fixed to point straight up from theground towards zenith. Each LOFAR station consists of a num-ber of such elements, each with the same alignment of xyz -axes.Other similar instruments, including LOFAR High Band An-tenna (HBA) stations, have similar overall structure, but maydi ff er in details.The telescope is electronically “steered” towards a target ina reference direction w by combining signals from each ele-ment with appropriate delays. Since the w -axis can be chosenarbitrarily, the angle between it and the x -axis can be seen tovary independently to the angle between it and the y -axis.The relative geometric orientation of the inbound radiationfrom a source to that of a dipole antenna leads to a variation inthe voltage response of that antenna. A simple element of this this is the same w from which the uvw -coordinates are calculated for imag-ing studies a r X i v : . [ a s t r o - ph . I M ] A ug . Creaner, & T. Carozzi / Astronomy and Computing 00 (2019) 1–10 variation comes from the fact that E-M waves are transverse,and because of this, the sensitivity is maximised when the w -axis is orthogonal to the antenna, and minimised when the w -axis is parallel with the antenna. Because for LOFAR, the z -axis is fixed towards zenith, the sensitivity is maximised for asource at zenith while at lower altitudes, this sensitivity can bereduced substantially as the angle between the source and theantenna decreases. Since all antennas in a station are alignedon the same axes, this antenna-level e ff ect could be expected toimpact the station-level response as well (Carozzi, 2017).As an object rotates about the celestial pole over the courseof a sidereal day, there is a separate variation for each of thetwo orthogonal antennas. This is because the angle betweenthe source and the linear antennas can widen and narrow sepa-rately, changing the response of each antenna. This can lead toan apparent polarisation in the detected signal. Again, this con-trasts with mechanically steered telescopes where the axes ofthe telescope can be rotated to follow the source and maintainmaximum sensitivity in all directions. Additional complexitiesfor modelling this variation occur as signals interact with multi-ple antennas in a station, and atmospheric e ff ects are taken intoconsideration (Di Ninni et al., 2019). Figure 1. A schematic representation of a single LBA element of a LOFARstation in Elevation and Plan view. The element consists of a ground plane(represented in black) and two identical antennas (represented in red and blue)supported at the centre of the element and orthogonal to one another. Eachantenna defines an axis along its length ( x -axis in red, y -axis in blue) and theaxis orthogonal to both defines a third axis z (shown in green). An arbitrarytarget in direction w can be selected such that ∠ wx and ∠ wy can independentlyvary between 0°and 90°. Note that in the Elevation view, the x -axis is viewedend-on, while in the Plan view, the z -axis is viewed end-on. Thus, there is a variation with respect to altitude (used in this paper in the sense of angle above the horizon) and azimuth(angle East / West of the Northern meridian) which must be mod-elled. For the study of flux- or polarisation-variability of ob-jects, it is essential to correct for this instrumental variation bymeans of calibration. In the case of LOFAR, these calibrationsare carried out based on an analytical model, such as that ini-tially developed by Hamaker (2011) which is integrated into thedata processing software (e.g. Default Pre-Processing Pipeline(DPPP) (Dijkema et al., 2008–)). These models are discussedin more detail in Section 3. beamModelTester is designed to compare the predictionsof these models with outputs from real observations to providean assessment of the quality of these models. Throughout thispaper, plots are shown that provide examples of this softwaresystem in operation, which include automatically generated ti-tles and axis labels. These automatically generated plots areshown with a black outline.
2. Sample Reference Measurement
In order to provide a demonstration of the system in oper-ation, it was necessary to produce an observation and a modelwhich could be compared with one another using the softwarepresented here. The requirement for the observation was that itbe of a bright source which was circumpolar from the availableobservatory, ideally passing close to zenith. The source was re-quired to be point-like at the resolution of the instrument thatwas used.
Figure 2. A plot of observed flux (Stokes I) from CasA against Time and Fre-quency as observed with LOFAR Station SE607 HBA over 12 hours. This plothas been trimmed to remove RFI-dominated frequencies and normalised acrossfrequencies using the maximum method. Plot (including title and axis labels)automatically generated using beamModelTester . . Creaner, & T. Carozzi / Astronomy and Computing 00 (2019) 1–10 A 24-hour observation of the radio source Cassiopeia A(CasA) taken on 16th-17th March 2018 from LOFAR HBA sta-tion SE607 at Onsala Space Observatory, Sweden is used a sam-ple source to demonstrate this software in operation through-out this paper. The timescale of variation in flux from CasA isknown to be very much longer than the 24 hours over which theobservation took place (Helmboldt and Kassim, 2009).The station is located at Latitude: 57°23 (cid:48) (cid:48)(cid:48) N, Longi-tude: 11°55 (cid:48) (cid:48)(cid:48) E. The HBA consists of 96 HBA tiles, eachof which consists of 16 pairs of orthogonal antennas. Each tilefeeds into two Receiver Control Units (RCUs), one for eachpolarisation. The maximum separation between antennas in theHBA field is 60m. The maximum frequency in this observationis 200MHz, giving a minimum wavelength of 1.5m. Therefore,the maximum resolution is ∼ ∼ (cid:48) , (Arias et al., 2018)and thus can be treated as a point source for the purposes of thisexperiment.The observation consisted of recording a series of array co-variance matrices (ACMs) also known as “crosslets” or “visi-bilities,” at a rate of 1 per second (Virtanen, 2012). These werestored in Array Covariance Cubes (ACC Files). Each ACC Fileconsists of 512 ACMs, one for each subband in the HBA range.A 5 second interval between the end of one ACC recording andthe start of the next gave an observation cadence of one com-plete observation of the frequency range of the HBA every 519seconds. These ACMs were converted into beamformed ob-servations of a point source using iLiSA (Carozzi, 2018–) asdiscussed in Section 5. Because this 519s observation cadenceis short compared with the 24-hour runtime of the observation,observations in a single ACC file are treated as taking place si-multaneously.Figure 2 demonstrates the variation of the apparent flux(Stokes I: calculated as the sum of the x - and y -axis fluxes)from CasA over the course of this observation. This variationwith respect to orientation is not fully consistent with respectto frequency. While the general trend remains for intensity ineach subband to reach a maximum at about the same time anda minimum at about the same time, the curve between thesepoints has noticeably di ff erent shapes at di ff erent frequencies.The apparent variation in CasA is believed to be induced by itschanging orientation with respect to the antennas.By taking the position of the station, the target and the time,it is possible to calculate the location of the target on the skyat any time as discussed in Section 5. Figure 3 shows the pathtaken by CasA on the sky during the 24-hour observation. FromSE607, CasA reaches a maximum altitude of 88.58° (i.e. veryclose to zenith) and the lowest altitude of 26.24°.An example of the variation in flux with respect to altitudecan be seen in Figure 4. For this plot, the altitude and azimuthare calculated for each of the time steps and the former is usedas an axis for plotting. As is shown, the flux varied by a factorof up to ∼ Figure 3. Path of CasA on the sky as observed from LOFAR station SE607 over24 hours, passing just North of zenith at a maximum altitude of 88.58°. Plotgenerated using dreamBeam . This output displays the position in zenith angle(90°- altitude) and azimuth.Figure 4. A plot of observed flux (Stokes I) from CasA against Alti-tude as observed with LOFAR Station SE607 HBA at 162.5MHz over 24hours. Plot (including title and axis labels) automatically generated using beamModelTester . altitude changes lead to a change in both x- and y-antenna re-sponse, azimuth changes lead to a di ff erent response for each ofthe linear antennas. This means that an apparent linear polarisa-tion (Stokes Q: calculated as the di ff erence between the x - and y -axis fluxes) will arise in a system with linear antennas whichdo not move as the source azimuth to the detector changes withthe rotation of the earth as shown in Figure 5.Figure 5 demonstrates the major trend in apparent polari-sation over azimuth can be seen in form of a peak at azimuth ∼ ∼ . Creaner, & T. Carozzi / Astronomy and Computing 00 (2019) 1–10 which is discussed briefly in Section 8. Figure 5. (top) A plot of observed linear polarisation (Stokes Q) against Az-imuth and Frequency. (bottom) A plot of Altitude against Azimuth at the cor-responding times of this observation to indicate the variation in two indepen-dent variables. This plot has been trimmed to remove RFI-dominated frequen-cies. Paired plot (including title and axis labels) automatically generated using beamModelTester . Since response in flux and polarisation can clearly be seento vary against frequency, time, altitude and azimuth, it followsthat any model developed must be able to incorporate each ofthose variables.A model of this observation was generated using dreamBeam (Carozzi, 2016–). dreamBeam is a piece of software designed toimplement existing models of telescope performance. dreamBeam produces predictions of the flux from an ideal point-like sourceat given sky coordinates as observed from a given observing lo-cation. None of the models currently implemented in dreamBeam incorporate sky models. As used in this system it outputs thepredictions of the model to a
CSV file.
3. Existing Models
A mathematical framework developed by J. P. Hamaker pro-vides a mathematical-physical basis for the description of thevariation in the response of a LOFAR station based on a mixof analytical and numerical simulations (Hamaker, 2011). Thecoe ffi cients for this framework were implemented by M. Arts.This framework forms part of DPPP (Dijkema et al., 2008–;Shulevski et al., 2019). An implementation of this pipeline isused in dreamBeam to calculate predicted values for observa-tions (Carozzi, 2016–).This model is known to have some limitations and reserva-tions which must be addressed (Hamaker, 2011). Firstly, thesimulation is based on an extension of a model of a single an-tenna under ideal conditions i.e. away from electromagnetic obstructions (Hamaker, 2011). In order to produce an arraysuch as is used in LOFAR, such obstructions are all around inthe form of the other antennas in the array (Asad et al., 2015;Di Ninni et al., 2019). Secondly, the model, by design, is basedon the open-circuit voltage, i.e. it does not include impedancematching (Hamaker, 2011).The model, therefore, is known to be incomplete in its pre-dictions. Quantitative assessment of the degree to which thismodel diverges from practical observation are therefore requiredto enable users of the system to have confidence in its predic-tions.In addition, ongoing work on refined or other alternativemodels has been discussed by Asad et al. (2015, 2016) andDi Ninni et al. (2019) amongst others. To compare the per-formance of these models with the model used in the LOFARPipeline to determine which should be used it is necessary todevelop a robust structure to provide a figure of merit for theperformance of the model.Finally, it is hoped that by comparing models with observa-tions and with one another, it might be possible to guide furtherrefinements to the models by highlighting areas of the spectrumor sky in which the performance of the model is deficient.
4. Requirements for testing system
It is therefore apparent that any such system of modellingwould require a robust system to test and evaluate its perfor-mance and such a system is presented here. The key featuresof a testing system are that it be modular, flexible and user-friendly.In this context, modularity is essential under two categories.Data modularity is the requirement that the system must be ableto handle inputs from disparate systems - model data from eachof a variety of models, observation data from a variety of tele-scopes - where the data structure cannot be guaranteed to beuniform between di ff erent systems. Modules capable of loadingdata from each possible source are therefore necessary. Func-tional modularity is the design requirement that mandates thatcomponents of the system be able to work independently fromone another, which allows independent features to be added tothe system in response to user demand without compromisingexisting functionality.Flexibility is the requirement to provide immediate, tailoredoutputs in a variety of use cases, to enable the examination ofa variety of parameters of the models to be tested. Further,the capture of additional use cases for related objectives, suchas automated production of dynamic spectra can be achievedthrough flexible design.A user-friendly system is responsive to user demands, hasclear input mechanisms and gives outputs which are clear andinstantly recognisable. By focussing on human-centred design,the presentation of output results enables robust analysis for fol-low up studies.4 . Creaner, & T. Carozzi / Astronomy and Computing 00 (2019) 1–10
5. Software Solution Design
The solution presented here consists of three main modu-lar components, each designed to “plug-in” seamlessly to theothers, and which can be replaced with equivalent componentsfor future modules and applications. As shown in Figure 6, thethree major elements are model data calculation and input (cur-rently provided by dreamBeam , Carozzi (2016–)), observationdata recording and transformation (provided by iLiSA , Carozzi(2018–)) and a module to combine and compare data from eachof the input components.Model data is provided through dreamBeam , a multi-purposesystem which implements the beam models, such as the existingmodel as discussed above and, as used in this suite, producesoutputs in the form of text which can be stored as a
CSV file(Carozzi, 2016–, 2017). Inputs to dreamBeam specify a target,model, observing location and time for the observation. Theoutput is a normalised prediction of variation over time (andfrequency) for an uncontaminated point source in Jones Matrixform.Observed data is currently provided through iLiSA (Inter-national LOFAR in Stand Alone mode). This system has mul-tiple elements, but the feature used here is the ability to usecross-correlation files - specifically Array Covariance Cubes(
ACC files) - to generate beamformed fluxes for a given targetwithin the beam of the telescopes used. This calculation is car-ried out without assumptions regarding a sky model and thusdoes not include demixing of other sources. It also operates un-der the assumption that the target is point-like at the resolutionof the single station. Outputs from iLiSA in this mode consistof
HDF5 files containing the observed times, frequencies andlinear channel fluxes xx , xy and yy (Carozzi, 2018–).The testing system brings these two together by providingflexible plug-ins for the data to beamModelTester . The read-ing functions of this suite of software read the data from eachof the sources into a Pandas dataframe (McKinney, 2010).Linear fluxes and Stokes parameters are then calculated fromthe Jones Matrix data. Data from the two sources are joinedusing common variables date / time and frequency to allow toenable comparisons between the model and the observed data.Throughout this system, all times are assumed to follow thesame standard, usually UTC. Conversion between local timeand UTC is possible by adding an appropriate time o ff set aspart of the user input.These programs are tied together using two wrapper scripts.These scripts transfer the variables and data from one programto another. Note that separate scripts are provided to access datain raw form and data that has already been processed once. Dataprocessing from raw data takes time, but the intermediate dataformats ( CSV and
HDF5 files as shown in Figure 6) are muchfaster to load into memory than to generate from the sourceformat
ACC files.The Horizontal coordinates Altitude and Azimuth are calcu-lated from the station geographic coordinates and the celestialcoordinates of the target for each timestamp in the observationusing the coord.transform to method of
Astropy (The As-tropy Collaboration et al., 2013, 2018). An East-West version
Figure 6. Overall design of solution. dreamBeam (Carozzi, 2016–) providesmodel data of the predicted variation of a target over the course of an observa-tion in a
CSV file. iLiSA (Carozzi, 2018–) converts
ACC files output from thetelescope into a
HDF5 file with fluxes at given times and frequencies for a giventarget. The comparison module of beamModelTester (Creaner, 2018–) bringsthese together in memory for processing into charts. (as opposed to 0-360°) of Azimuth is calculated by subtracting360° from all Azimuths over 180°. A rotated set of coordi-nates called
Station coordinates are calculated from the Hori-zontal coordinates by means of the antennafieldlib methodof iLiSA (Carozzi, 2018–).
Station coordinates are rotated suchthat the x - and y -axes of a given station are aligned with the or-dinal (NE, SE, SW and NW) points of the Station coordinatesystem.This creates two sets of variables that can be compared,the independent variables and the dependent ones. The inde-pendent variables are time, frequency, altitude and azimuth,the latter of which can be in either conventional horizontal orstation relative coordinates. The dependent variables are a setof di ff erent formulations of measures of flux and polarisation.These consist of the linear antenna fluxes (xx and yy), the cross-correlation antenna component (xy) and the Stokes parameters(I, Q, U, V.)The system is designed for use from the command line viainput parameters, via a text-based menu interface or using agraphical user interface (GUI). Each of the interfaces enablesthe user to select the dependent and independent variable(s) toplot, as well as to select between various representation ap-proaches and output data types and locations. The code, to-gether with detailed instructions and tutorials on its use areavailable from Creaner (2018–).
6. Comparison Mechanisms
Comparisons between observation and model values can bemade by three main approaches, Direct Comparison, Di ff erencePlots and Figures of Merit.5 . Creaner, & T. Carozzi / Astronomy and Computing 00 (2019) 1–10 Direct Comparison.
Plots which show both model and observeddata and enable the user to compare the two while retainingclear view of the original data. • Separate 3-d colour and / or contour plots of the variationof a dependent variable against the independent variablescan be generated for the model and observation data to beused in side-by-side comparisons for 3-variable data (e.g.flux against frequencies and time, as shown in Figure 2) • Separate 3-d colour plots with context plot of the varia-tion of a dependent variable against the independent vari-ables can be generated for the model and observation datato be used in side-by-side comparisons for 4-variable data(e.g. flux against frequencies and azimuth with a contextplot showing altitude against azimuth as shown in Figure5) • Line plots of model and observed data can be generated toallow for direct comparison for 2-variable data (e.g. fluxagainst altitude at a fixed frequency as shown in Figure 4)which may be overlaid upon one another or shown side-by-side. • Animated line plots of model and observed data whereone independent variable is assigned to time, and anotherto the x-axis (e.g. showing how the apparent spectrumvaries over time.) Again, these can be overlaid or placedside-by-side. Di ff erence Plots. Plots of the di ff erence between the model andthe observation. These plots can be displayed with or withoutthe original data, and can be plotted in the same manner as thedirect plots above. Di ff erences can be calculated in the forms • Subtraction (model-observation) This approach determinesthe di ff erence between the model and the observation bysubtracting the value of the observation from that of themodel. In order for this to be a sensible approach, the datamust be in the same units and appropriately normalised.The output is viable even in cases where one or both val-ues are zero or negative • Division (model / observation) This approach determinesthe di ff erence between the model and the observation byreference to the observation. The variation in this valueindicates a variation in the ratio between the two mea-surements, and can be used without normalisation. Out-puts can be di ffi cult to interpret if there are negative val-ues for some measurements, and is undefined if the ob-served value, or both values are zero. • Inverse division (observation / model) This approach de-termines the di ff erence between the model and the obser-vation by reference to the model. The variation in thisvalue indicates a variation in the ratio between the twomeasurements, and can be used without normalisation.Outputs can be di ffi cult to interpret if there are negativevalues for some measurements, and is undefined if themodel, or both values are zero. Figures of Merit.
Calculations and plot of figures of merit forthe variation between model and observation, and how thosefigures vary across independent variables. The figures of meritused are Root Mean Square Error (RMSE) and Pearson’s Cor-relation. • RMSE
This figure of merit is a measure of the averageseparation of the model value from the observed value. Itremains valid even if some values are negative or zero. Inorder for this to be a sensible approach, the data must bein the same units and appropriately normalised. • Pearson’s Correlation
This is a measure of how similarthe pattern of variation in the observation matches thatin the model. This figure of merit can be used withoutnormalisation.
7. Cropping and Normalisation
In addition to the above mentioned plotting options, thereare two other factors which must be addressed when compar-ing observations with models: elimination of Radio FrequencyInterference (RFI), and ensuring comparable units.RFI can be a major issue, especially for instruments such asLOFAR HBA which operates in the same frequency bands asartificial sources such as radio communications (O ff ringa et al.,2013). Two approaches to eliminate RFI exist within the cur-rent software solution. Additionally, the modular nature of thesoftware enables the development of more sophisticated RFIfiltering options in the future if needed.The first method of removing RFI is a frequency filter. Thisfilter allows for only specified frequencies to be plotted in theoutput data. Known frequencies / subbands where RFI is strong(e.g. known Air Tra ffi c Control channels) can be removed froma list of subband start-points and input either manually or as afile containing a list of viable frequencies.The second method to eliminate RFI is cropping. The usercan specify thresholds above which data points are eliminated.This cropping can be operated globally, or on a per-frequency orper-time basis, can the threshold can be set using a percentile,or using a multiple of the median or mean values in the data.Cropping allows for automated elimination of outliers but suf-fers in that it can lead to the elimination of real data as well asRFI-driven outliers.To ensure that the model inputs are comparable to the in-puts from observations, the data must be normalized. In thissystem, various options for normalisation are provided, depend-ing on the requirements of the user. The normalisation systemis modular, and can be extended with additional options in fu-ture versions of the software. The current normalisation opera-tions are Maximum-based normalisation and Fit-based normal-isation, Maximum-based normalisation means to divide by themaximum value in each of the input columns in the dataset, be-fore calculating the derived values. Fit-based normalisation cal-culates the linear multiplication factor and constant o ff set thatprovides a least-square fit between the model and the observa-tion, and applies these factors to the observation.6 . Creaner, & T. Carozzi / Astronomy and Computing 00 (2019) 1–10 These normalisations can be applied to the data either over-all, on a per-frequency or a per-time basis. In overall normalisa-tion mode, the system applies the normalisation process to thewhole dataset at once. In frequency- and time-normalisationmode, the unique values of each of these independent variablesare determined. The normalisation operation is carried out onthe data corresponding to each of these values. Frequency modeenables comparison to be made e ff ectively between observa-tions which necessarily are dependent on the spectral responseof the telescope, and models of those variations which do notaccount for the overall sensitivity curve. Time-normalisation al-lows the user to examine variations in the shape of the detectedspectrum as the target rotates about the pole.
8. Results
This system has been fully implemented and is operational.As such, it enables users of radio telescopes with no movingparts such as LOFAR to calibrate for a variety of di ff erent con-ditions and allows for examination of many issues that can arisedue to the relative alignment of source and detector. Variable Description Dependent?xx x-axis flux Dependent xy x / y-axis cross-correlation* Dependent yy y-axis flux Dependent Stokes I xx + yy overall flux Dependent Stokes Q xx-yy linear polarisation Dependent
Stokes U
Real (xy) polarisation angle Dependent
Stokes V
Im (xy) circular polarisation Dependent
Frequency
Frequency of subband start Independent
Time
Time of the start of ACC file Independent
Altitude
Angle of target above Horizonas viewed from the station Independent
Azimuth
Angle of target East of Northfrom the station Independent
Azimuth(E / W) Angle of target East ( + ) or West(-) of North from the station Independent StationAltitude
Angle of target above Horizonin
Station coordinates Independent
StationAzimuth
Angle of target East of North in
Station coordinates Independent
StationAzimuth(E / W) Angle of target East ( + ) or West(-) of North in Station coordi-nates Independent
Table 1. A list of variables that can be plotted, with an indication as to whetherthey are treated as a dependent or independent variable. *xy is a complex value.When plotting, the absolute value is plotted.
For a given observation, each of the set of variables repre-senting flux and polarisation described in Section 5 and sum-marised on Table 1 can be plotted against a set of independentvariables: altitude, azimuth or time on one axis and frequencyon the other. These plots can be generated as 3-d contour orcolour plots, as animated plots or as individual frames depend-ing on the needs of the user. Outputs can be stored in any suit- able location, and a variety of file types are available, depend-ing on the user and the software environment. Data from sourceand model can be filtered, cropped and normalised together orseparately using a variety of options for each parameter. Plotscan be overlaid or plotted separately as needed to allow for di-rect comparisons or to clear up graphical plots. These plots aresummarised on Table 2.
Plot Variables Overlay?2-d line plot
Animated line plot & y -axes)1 Dep (z-axis) No Table 2. The Plot column shows a list of the types of plots that can be generated.The Variables column shows the types of variables (i.e. Independent / DependentVariables) that can be plotted, based on Table 1 and the axis they are mappedto. The Overlay column indicates whether multiple plots can be shown overlaidupon one another for plots of a given type. (e.g. as shown in Figure7)Figure 7. A plot of observed and model flux in the y -polarisation channelfrom CasA against Altitude as observed with LOFAR Station SE607 HBA at131.445MHz over 24 hours. Illustration of an area of the spectrum where themodel and the observation are in close agreement. At higher altitude, noiselevels are greater than the model-source disagreement. Plot (including title andaxis labels) automatically generated using beamModelTester . Taken together, these parameters allow for many thousandsof possible combinations and permutations for a given set ofinput data depending in the use-case required, and it would beimpossible to fully discuss them all here. A number of sampleobservations which demonstrate the utility of the system areoutlined below.Figures 7 and 8 each show the comparison between the7 . Creaner, & T. Carozzi / Astronomy and Computing 00 (2019) 1–10 x -polarisation channelfrom CasA against Altitude as observed with LOFAR Station SE607 HBAat 187.500MHz over 24 hours. Illustration of an area of the spectrum wherethe model and observed data deviate strongly from one another. Here, nei-ther the general shape of the distribution nor the specific values are in agree-ment. Plot (including title and axis labels) automatically generated using beamModelTester . dreamBeam output (labelled model) and the iLiSA output (la-belled observed) as well as the di ff erences between them. Over-laid plots such as these, which are taken from frames from ananimated plot of this comparison, allow for areas in the dis-tribution where model and source diverge significantly to beobserved, to enable modellers to understand which regions tofocus on when refining their models. For example, in Figure7, especially at high altitudes, the model and observation arein close agreement, and thus the di ff erence can be shown to below. On the contrary, in Figure 8, the shape and position of theobserved curve is significantly di ff erent to the model. This sug-gests a region of the radio spectrum in which the model shouldbe modified to account for additional factors.Plotting the data in three dimensions can allow for patternsto be noted in the distribution of divergences between the modeland the observation. As is shown in the upper plot of Figure 9,there are a number of smooth curved shapes which have beenhighlighted. Tasse et al. (2012) suggests some explanationswhich can be applied to features such as these. For example:many can be explained by the non-point-like behaviour of radiotelescope beams. One explanation is that features like these canbe caused by the side-lobes of the LOFAR beam shape coveringanother object in the so-called “A-team” set of sources.In particular, LOFAR HBA observations are vulnerable tothe presence of bright objects in the side lobes of the beam pat-tern, as the regular and even spacing of HBA antennas can leadto extreme sidelobe patterns such as that shown in Figure 10.The shape of these patterns is dependent on the uv spacing ofthe antenna elements, which will necessarily change as the w-axis changes to follow the observed object. Additionally, asthe size, strength and spacing of these sidelobes is wavelength- Figure 9. (top) A plot of the di ff erence between observed and model of linearpolarisation (Stokes Q) against Azimuth and Frequency. (bottom) A plot of Al-titude against Azimuth at the corresponding times of this observation to indicatethe variation in two independent variables. Note the parabola-like contamina-tion from sidelobe observations of other A-Team sources. Paired plot (includingtitle and axis labels) automatically generated using beamModelTester . Redcurves manually added to draw attention to patterns in the data. dependent, cross-contamination will not necessarily occur atthe same time, altitude or azimuth for all frequencies as illus-trated by the di ff erences between Figures 10 and 11. As a re-sult, the additional contaminated flux can appear to move acrosswavelengths as the target source moves across the sky. Thisis believed to produce the distinctive parabola-like structuresshown in Figure 9. Figure 10. Orthographic projection model of the Array Factor of LOFAR HBAstation SE607 at 120MHz. Note the extensive sidelobe patterns reaching all theway to the horizon. Plot generated using dreamBeam . It is apparent from Figure 9 that the model output from8 . Creaner, & T. Carozzi / Astronomy and Computing 00 (2019) 1–10 ff erences in the beampattern from thatshown in Figure 10 In particular, note the narrower spacing of the fringes andthe strong local maximum located close to the horizon. Plot generated using dreamBeam . the implementation of the Hamaker model in dreamBeam doesnot account for features such as these. Should a model beavailable to account for these deviations, it can be tested with beamModelTester and a reduced di ff erence between the modeland the observation would be expected in the correspondingplots.
9. Conclusions
Use of this software system enables a user wishing to quan-tify the performance of a model of radio telescopes with nomoving parts to robustly compare the model with a real obser-vation. By plotting the observation alongside the model and / orplotting the di ff erence between the observation and model, theuser can determine any areas where the model does not give anaccurate representation of real observations. This can be used toidentify additional factors, such as cross-contamination by sec-ond (and more) sources in the side-lobes of a beam which mustbe accounted for in any attempt to calibrate the observation bymeans of a model.The design of the software solution enables it to be extendedto use additional inputs with minimal modification to the sys-tem. Notably, if new models are developed which are intendedto more closely describe expected observations, these can beeither plugged in directly, if their output format matches an ex-isting format, or a suitable plugin developed to import the data,which can then be compared against observation using the ex-isting system.Ongoing refinements to the system are envisaged to take theform of upgrades, rather than invalidating existing use of thesystem. Planned upgrades include the ability to integrate mul-tiple targets or observations from multiple stations to providemore complete sky coverage.The software system has been successfully designed, im-plemented, and is available together with user and design docu- mentation at the link below: A large selection of sample outputs are available at the linkbelow.
Acknowledgements
This publication has received funding from the EuropeanUnions Horizon 2020 research and innovation programme un-der grant agreement No 730562 [RadioNet].The software developed in this project makes use of the fol-lowing software packages: • dreamBeam : a Radio telescope beam modeling frame-work (Carozzi, 2016–) • iLiSA : international LOFAR in Stand-Alone mode (Carozzi,2018–) • Matplotlib : A 2D graphics environment (Hunter, 2007) • Astropy : a community-developed core Python packagefor Astronomy (The Astropy Collaboration et al., 2013,2018), • python-casacore : A wrapper around CASACORE, theradio astronomy library (van Diepen et al., 2007) • KERN : a bi-annually released set of radio astronomicalsoftware packages (Molenaar and Smirnov, 2018) • pandas : Python Data Analysis Library (McKinney, 2010) • NumPy : the fundamental package for scientific computingwith Python (Van Der Walt et al., 2011) • SciPy : Open source scientific tools for Python (Joneset al., 2001–) • H5Py : HDF5 for Python (Collette et al., 2014–; Collette,2013)
References
Arias, M., Vink, J., De Gasperin, F., Salas, P., Oonk, J., Van Weeren, R.,Van Amesfoort, A., Anderson, J., Beck, R., Bell, M., et al., 2018. Low-frequency radio absorption in Cassiopeia A. Astronomy & Astrophysics612, A110.Asad, K., Koopmans, L., Jeli´c, V., Ghosh, A., Abdalla, F., Brentjens, M.,de Bruyn, A., Ciardi, B., Gehlot, B., Iliev, I., et al., 2016. Polarizationleakage in epoch of reionization windows–II. Primary beam model anddirection-dependent calibration. Monthly Notices of the Royal Astronomi-cal Society 462, 4482–4494.Asad, K., Koopmans, L., Jeli´c, V., Pandey, V., Ghosh, A., Abdalla, F., Bernardi,G., Brentjens, M., de Bruyn, A., Bus, S., et al., 2015. Polarization leakagein epoch of reionization windows–I. Low Frequency Array observations ofthe 3C196 field. Monthly Notices of the Royal Astronomical Society 451,3709–3727.Butcher, H.R., 2004. LOFAR: First of a new generation of radio telescopes, in:Ground-based Telescopes, International Society for Optics and Photonics.pp. 537–545. . Creaner, & T. Carozzi / Astronomy and Computing 00 (2019) 1–10 . [Online; accessed2019-06-04].Carozzi, T.D., 2017. dreamBeam: A software package for radio telescopebeam models with a LOFAR plugin. . [Online; accessed 2019-06-04].Carozzi, T.D., 2018–. iLiSA - international LOFAR in stand-alone mode. . [Online; accessed 2019-06-04].Collette, A., 2013. Python and HDF5: Unlocking Scientific Data. O’ReillyMedia, Inc.Collette, A., et al., 2014–. HDF5 for Python. . [Online; ac-cessed 2019-06-04].Creaner, O., 2018–. beammodeltester: a general-purpose tool for evaluationof models of radio telescope beams. . [Online; accessed 2019-06-04].Di Ninni, P., Bolli, P., Paonessa, F., Pupillo, G., Virone, G., Wijnholds, S.J.,2019. Electromagnetic analysis and experimental validation of the LOFARradiation patterns. International Journal of Antennas and Propagation 2019.van Diepen, G., Marquarding, M., Molenaar, G., 2007. Python bindings forcasacore, a library used in radio astronomy. .Dijkema, T.J., van Diepen, G., O ff ringa, A., Loose, M., Schoenmakers, A.R.A.,Mol, J.D., Grange, Y., pdonker, Molenaar, G., Ra ff erty, D., 2008–. Dp3:streaming processing pipeline for radio interferometric data. . [Online; accessed 2019-06-20].Hamaker, J., 2011. Mathematical-physical analysis of the generic dual-dipoleantenna. Technical Report. Tech. rep., ASTRON.Helmboldt, J., Kassim, N., 2009. The evolution of cassiopeia a at low radiofrequencies. The Astronomical Journal 138, 838.Hunter, J.D., 2007. Matplotlib: A 2d graphics environment. Computing inScience & Engineering 9, 90–95.Jones, E., Oliphant, T., Peterson, P., et al., 2001–. SciPy: Open source scientifictools for Python. . [Online; accessed 2019-06-04].McKinney, W., 2010. Data Structures for Statistical Computing in Python ,in: van der Walt, S., Millman, J. (Eds.), Proceedings of the 9th Python inScience Conference , pp. 51 – 56.Molenaar, G., Smirnov, O., 2018. KERN. Astronomy and Computing .O ff ringa, A., De Bruyn, A., Zaroubi, S., van Diepen, G., Martinez-Ruby, O.,Labropoulos, P., Brentjens, M.A., Ciardi, B., Daiboo, S., Harker, G., et al.,2013. The LOFAR radio environment. Astronomy & astrophysics 549, A11.Shulevski, A., Pizzo, R.F., van Diepen, G., Dijkema, T.J., Heald, G.,de Gasperin, F., Iacobelli, M., McKean, J., Mevius, M., O ff ringa, A., Orr´u,E., Ra ff erty, D., Tasse, C., van der Tol, B., Vacca, V., Vilchez, N., vanWeeren, R., Williams, W., Yatawatta, S., 2019. LOFAR imaging cookbook. support.astron.nl/LOFARImagingCookbook . [Online; accessed 2019-25-19].Tasse, C., van Diepen, G., van der Tol, S., van Weeren, R.J., van Zwieten,J.E., Batejat, F., Bhatnagar, S., van Bemmel, I., Bˆırzan, L., Bonafede, A.,et al., 2012. LOFAR calibration and wide-field imaging. Comptes RendusPhysique 13, 28–32.The Astropy Collaboration, Price-Whelan, A., Sip˝ocz, B., G¨unther, H., Lim, P.,Crawford, S., Conseil, S., Shupe, D., Craig, M., Dencheva, N., et al., 2018.The Astropy project: Building an inclusive, open-science project and statusof the v2. 0 core package. arXiv .The Astropy Collaboration, Robitaille, T.P., Tollerud, E.J., Greenfield, P.,Droettboom, M., Bray, E., Aldcroft, T., Davis, M., Ginsburg, A., Price-Whelan, A.M., Kerzendorf, W.E., et al., 2013. Astropy: A communityPython package for astronomy. Astronomy & Astrophysics 558, A33.Van Der Walt, S., Colbert, S.C., Varoquaux, G., 2011. The NumPy array:a structure for e ffi cient numerical computation. Computing in Science &Engineering 13, 22.Virtanen, I., 2012. Station data cookbook. . [Online; ac-cessed 2019-06-19].. [Online; ac-cessed 2019-06-19].