Jim Duggan

Applying reinforcement learning towards automating resource allocation and application scalability in the cloud

Public Infrastructure as a Service (IaaS) clouds such as Amazon, GoGrid and Rackspace deliver computational resources by means of virtualisation technologies. These technologies allow multiple independent virtual machines to reside in apparent isolation on the same physical host. Dynamically scaling applications running on IaaS clouds can lead to varied and unpredictable results because of the performance interference effects associated with co‐located virtual machines. Determining appropriate scaling policies in a dynamic non‐stationary environment is non‐trivial. One principle advantage exhibited by IaaS clouds over their traditional hosting counterparts is the ability to scale resources on‐demand. However, a problem arises concerning resource allocation as to which resources should be added and removed when the underlying performance of the resource is in a constant state of flux. Decision theoretic frameworks such as Markov Decision Processes are particularly suited to decision making under uncertainty. By applying a temporal difference, reinforcement learning algorithm known as Q‐learning, optimal scaling policies can be determined. Additionally, reinforcement learning techniques typically suffer from curse of dimensionality problems, where the state space grows exponentially with each additional state variable. To address this challenge, we also present a novel parallel Q‐learning approach aimed at reducing the time taken to determine optimal policies whilst learning online. Copyright

A Tool to Support Collaborative Software Requirements Management

The system requirements specification (SRS) is a highly dynamic document that grows and evolves throughout a software development project, and it is critical that it be carefully engineered and managed. Because the SRS fulfils many roles and is of interest to a diversity of stakeholders, its management should be a collaborative process supported by an automated tool. Commercial requirements management tools are at present insufficiently versatile to support collaboration between a multidisciplinary and potentially distributed team of stakeholders. The requirements for such a collaborative tool are herein presented, alongside the design of a prototype and the findings of its application in a case study.

A task allocation optimizer for software construction

Task allocation during the construction stage of software engineering is complex and challenging. First, engineers must chart a path between the often conflicting objectives of time and quality. Second, a huge productivity variance exists across the spectrum of practicing software developers. Properly handling this variance amid those time and quality pressures is a tricky management problem. Multiobjective optimization might provide the answer. This emerging research area generates optimal solutions for projects with many objectives. An experienced decision-maker analyzes these solutions and selects the best one. Here, we describe such an approach and demonstrate it with a problem involving the allocation of software construction tasks among a team of software developers with varying degrees of skill.

A Learning Architecture for Scheduling Workflow Applications in the Cloud

The scheduling of workflow applications involves the mapping of individual workflow tasks to computational resources, based on a range of functional and non-functional quality of service requirements. Workflow applications such as scientific workflows often require extensive computational processing and generate significant amounts of experimental data. The emergence of cloud computing has introduced a utility-type market model, where computational resources of varying capacities can be procured on demand, in a pay-per-use fashion. In workflow based applications dependencies exist amongst tasks which requires the generation of schedules in accordance with defined precedence constraints. These constraints pose a difficult planning problem, where tasks must be scheduled for execution only once all their parent tasks have completed. In general the two most important objectives of workflow schedulers are the minimisation of both cost and make span. The cost of workflow execution consists of both computational costs incurred from processing individual tasks, and data transmission costs. With scientific workflows potentially large amounts of data must be transferred between compute and storage sites. This paper proposes a novel cloud workflow scheduling approach which employs a Markov Decision Process to optimally guide the workflow execution process depending on environmental state. In addition the system employs a genetic algorithm to evolve workflow schedules. The overall architecture is presented, and initial results indicate the potential of this approach for developing viable workflow schedules on the Cloud.

Using semantic rules to determine access control for web services

Semantic Web technologies are bring increasingly employed to solve knowledge management issues in traditional Web technologies. This paper follows that trend and proposes using Semantic rule languages to construct rules for defining access control rules for Web Services. Using these rules, a system will be able to manage access to Web Services and also the information accessed via these services.

Stream enhancements for the CORBA event service

This paper describes a number of enhancements for the standard CORBA Event Service. The basic service definition has been extended to support stream events, multimedia data flows, event fragmentation, quality of service definition, as well as multicast event delivery. The paper evaluates the service performance and describes experiences using the enhanced service in the development of a test application.

An Experimental Review of Reinforcement Learning Algorithms for Adaptive Traffic Signal Control

Urban traffic congestion has become a serious issue, and improving the flow of traffic through cities is critical for environmental, social and economic reasons. Improvements in Adaptive Traffic Signal Control (ATSC) have a pivotal role to play in the future development of Smart Cities and in the alleviation of traffic congestion. Here we describe an autonomic method for ATSC, namely, reinforcement learning (RL). This chapter presents a comprehensive review of the applications of RL to the traffic control problem to date, along with a case study that showcases our developing multi-agent traffic control architecture. Three different RL algorithms are presented and evaluated experimentally. We also look towards the future and discuss some important challenges that still need to be addressed in this field.

An mHealth Intervention Using a Smartphone App to Increase Walking Behavior in Young Adults: A Pilot Study

Background Physical inactivity is a growing concern for society and is a risk factor for cardiovascular disease, obesity, and other chronic diseases. Objective This study aimed to determine the efficacy of the Accupedo-Pro Pedometer mobile phone app intervention, with the goal of increasing daily step counts in young adults. Methods Mobile phone users (n=58) between 17-26 years of age were randomized to one of two conditions (experimental and control). Both groups downloaded an app that recorded their daily step counts. Baseline data were recorded and followed-up at 5 weeks. Both groups were given a daily walking goal of 30 minutes, but the experimental group participants were told the equivalent goal in steps taken, via feedback from the app. The primary outcome was daily step count between baseline and follow-up. Results A significant time x group interaction effect was observed for daily step counts (P=.04). Both the experimental (P<.001) and control group (P=.03) demonstrated a significant increase in daily step counts, with the experimental group walking an additional 2000 steps per day. Conclusions The results of this study demonstrate that a mobile phone app can significantly increase physical activity in a young adult sample by setting specific goals, using self-monitoring, and feedback.

Production activity control: A practical approach to scheduling

There is a widely perceived gap within the domain of scheduling for manufacturing systems, namely, many of the methods employed by production supervisors are quite different from those developed by researchers. In a sense, this inconsistency highlights the important fact that much scheduling research has failed to win approval where it matters most, namely, within the manufacturing system.In this article, we argue for a practical approach to scheduling for manufacturing systems, one that we believe can narrow, and possibly bridge, the gap between theory and practice. This approach is based upon a well-defined and modular architecture for scheduling, termedproduction activity control. This architecture is the foundation of our proposed solution to scheduling, since it provides a coherent blueprint for the synthesis of information technology and scheduling strategies. The result of this synthesis is a design tool for production activity control, which allows for detailed and disciplined experimentation with a range of scheduling strategies in a controlled and simulated environment. Due to the unique modular property of the design tool, these strategies may then be implemented live in a flexible manufacturing facility, hence narrowing the gap between scheduling theory and manufacturing practice. Our overall approach is tested through an appropriate implementation in a modern electronics assembly plant.

Participatory Syndromic Surveillance of Influenza in Europe

The growth of digital communication technologies for public health is offering an unconventional means to engage the general public in monitoring community health. Here we present Influenzanet, a participatory system for the syndromic surveillance of influenza-like illness (ILI) in Europe. Through standardized online surveys, the system collects detailed profile information and self-reported symptoms volunteered by participants resident in the Influenzanet countries. Established in 2009, it now includes 10 countries representing more than half of the 28 member states of the European Union population. The experience of 7 influenza seasons illustrates how Influenzanet has become an adjunct to existing ILI surveillance networks, offering coherence across countries, inclusion of nonmedically attended ILI, flexibility in case definition, and facilitating individual-level epidemiological analyses generally not possible in standard systems. Having the sensitivity to timely detect substantial changes in population health, Influenzanet has the potential to become a viable instrument for a wide variety of applications in public health preparedness and control.