David G. Belanger

Virtual Data Warehousing, Data Publishing, and Call Detail

Many telecommunication applications can be modeled as data flowing through a network of systems. For example, the billing factory can be thought of as records of telephone calls flowing from one process to the next, as in from recording to rating to rendering. Probes can be attached at various points in the network. On the basis of these probes we would like to reason about the flow as a whole. Are there any losses? If so, where? How long does it take for a record to flow from one step to the next? Where are the bottlenecks? Two systems are described in a dataflow context: (1) Gecko, a system monitoring a very large billing system and (2) Pzip: a general compression tool based on gzip, which is typically twice as good as gzip in both space and time for sequences of fixed-length records.