Andrew McCarren

Anomaly detection in agri warehouse construction

As with many sectors, strategists and decision makers in the agricultural sector have a requirement to predict key measures such as product and feed pricing in order to maintain their position and, in some cases, to survive in their industry. Predictive algorithms in the area of Agri Analytics have shown to be very difficult due to the wide range of parameters and often unpredictable nature of some of these variables. Improving the predictive capability of Agri planners requires access to up-to-date external information in addition to the analyses provided by their own in-house databases. This motivates the need for an Agri Data Ware-house together with appropriate cleaning and transformation processes. However, with the availability of rich and wide ranging sources of Agri data now available online, there is a strong motivation to process as much current, online information as possible. In this work, we introduce the Agri Data Warehouse built for the DATAS project which not only harvests from a large number of online sources but also adopts an anomaly detection and labelling process to assist transformation and loading into the warehouse.

Combining Web and Enterprise Data for Lightweight Data Mart Construction

The Agri sector has shown an exponential growth in both the requirement for and the production and availability of data. In parallel with this growth, Agri organisations often have a need to integrate their in-house data with international, web-based datasets. Generally, data is freely available from official government sources but there is very little unity between sources, often leading to significant manual overhead in the development of data integration systems and the preparation of reports. While this has led to an increased use of data warehousing technology in the Agri sector, the issues of cost in terms of both time to access data and the financial costs of generating the Extract-Transform-Load layers remain high. In this work, we examine more lightweight data marts in an infrastructure which can support on-demand queries. We focus on the construction of data marts which combine both enterprise and web data, and present an evaluation which verifies the transformation process from source to data mart.

Data Mining in Agri Warehouses Using MODWT Wavelet Analysis

Agri-data analysis is growing rapidly with many parts of the agri-sector using analytics as part of their decision making process. In Ireland, the agri-food sector contributes significant income to the economy and agri-data analytics will become increasingly important in terms of both protecting and expanding this market. However, without a high degree of accuracy, predictions are unusable. Online data for use in analytics has been shown to have significant advantages, mainly due to frequency of updates and to the low cost of data instances. However, agri decision makers must properly interpret fluctuations in data when, for example, they use data mining to forecast prices for their products in the short and medium term. In this work, we present a data mining approach which includes wavelet analysis to provide more accurate predictions when events which may be classified as outliers are instead patterns representing events that may occur over the duration of the data stream used for predictions. Our evaluation shows an improvement over other uses of wavelet analysis as we attempt to predict prices using agri-data.

Variable interactions in risk factors for dementia

Current estimates predict 1 in 3 people born today will develop dementia, suggesting a major impact on future population health. As such, research needs to connect specialist clinicians, data scientists and the general public. The In-MINDD project seeks to address this through the provision of a Profiler, a socio-technical information system connecting all three groups. The public interact, providing raw data; data scientists develop and refine prediction algorithms; and clinicians use in-built services to inform decisions. Common across these groups are Risk Factors, used for dementia-free survival prediction. Risk interactions could greatly inform prediction but determining these interactions is a problem underpinned by massive numbers of possible combinations. Our research employs a machine learning approach to automatically select best performing hyperparameters for prediction and learns variable interactions in a non-linear survival-analysis paradigm. Demonstrating effectiveness, we evaluate this approach using longitudinal data with a relatively small sample size.