Hans-Arno Jacobsen

Processing Big Events with Showers and Streams

Emerging use cases derived from the area of cloud computing, smart power grids, and business process management require a set of capabilities not met by traditional event processing systems. These use cases were chosen to illustrate the capabilities required from systems that are able to process what we refer to as Big Events, that is Big Data in motion. To further illustrate Big Events, we identify three use cases and analyze the characteristics of the events involved. Based on this analysis, we specify requirements regarding the event schema, event query language, historic event processing needs, event timing, and result accuracy. Collectively, we refer to the constellation of state changes in a given system that exhibits these characteristics as event showers, referring to the collective of these events, similar to the notion of an event stream in the context of event stream processing. We call systems that offer capabilities for meeting these requirements event shower processing systems in contrast to traditional event stream processing systems. The use cases we picked, demonstrate that additional value can be captured by having shower processing systems in place. The benefits lie in the new possibilities to gain additional insights, increase observability, and to further exert control and opportunities for optimizations in the given applications.

A serverless topic-based and content-based pub/sub broker: demo

Building scalable, highly available publish/subscribe (pub/sub) systems can require sophisticated algorithms and a tremendous amount of engineering effort. This paper demonstrates a way to build a pub/sub broker on top of the OpenWhisk serverless platform that performs topic-based and content-based matching. This approach radically simplifies the design and significantly reduces the amount of code while still achieving scalability targets. Furthermore, we present a publisher/subscriber client application to interact with the broker as well as an evaluator application that enforces heavy workload on the broker to measure the scalability and latency of the pub/sub system and discover the potential bottlenecks.