AnyDB: An Architecture-less DBMS for Any Workload
AAnyDB: An Architecture-less DBMS for Any Workload
Tiemo Bang
TU Darmstadt & SAP SE
Norman May
SAP SE
Ilia Petrov
Reutlingen University
Carsten Binnig
TU Darmstadt
Abstract
In this paper, we propose a radical new approach for scale-outdistributed DBMSs. Instead of hard-baking an architectural model,such as a shared-nothing architecture, into the distributed DBMSdesign, we aim for a new class of so-called architecture-less DBMSs.The main idea is that an architecture-less DBMS can mimic anyarchitecture on a per-query basis on-the-fly without any additionaloverhead for reconfiguration. Our initial results show that ourarchitecture-less DBMS AnyDB can provide significant speed-upsacross varying workloads compared to a traditional DBMS imple-menting a static architecture.
Motivation:
Scale-out distributed architectures are used today bymany academic and commercial database systems such as SAPHANA, Amazon Redshift / Aurora, and Snowflake [4, 5, 8] to pro-cess large data volumes since they allow scaling compute and mem-ory capacities by simply adding or removing processing nodes. Thetwo predominant architectural models used in academic and com-mercial distributed databases are the shared-nothing (aggregated)architecture and the shared-disk (disaggregated) architecture.While the shared-nothing (aggregated) architecture provideshigh performance in case the data and workload are well partition-able, its performance degrades significantly under skew, since someresources are overloaded while others are idle. Moreover, dealingwith requirements such as elasticity in the shared-nothing archi-tecture is hard, since this always requires repartitioning the dataeven if compute is the bottleneck. This renders the shared-nothingarchitecture less suited for modern environments such as the cloudwhere elasticity is a key requirement [7].On the other hand, the shared-disk (disaggregated) architecturetackles the drawbacks of the shared-nothing architecture by sepa-rating storage from compute. This separation provides many newpotentials especially for better skew handling as well as providingelasticity independently for compute and storage. Yet, the shared-disk (disaggregated) architecture has other downsides. One majordownside is that data always needs to be pulled into the computelayer, resulting in higher latencies. While this additional latencyoften does not matter for OLAP, it renders this architecture less in-teresting for OLTP which requires low latency execution to reducethe potential of conflicts and provide high throughput.Another observation is that these architectural models (shared-nothing or shared-disk) are statically baked into the system designof today’s databases [2]. However, modern workloads are versatile,e.g., HTAP containing a mix of OLTP and OLAP queries. Moreover,workloads are often evolving over time or as in the cloud are noteven foreseeable for cloud providers, since workloads are user-specific. Hence, databases following a static architectural model arealways a kind-of-a compromise, since they cannot provide optimalperformance across a wide-spectrum of workloads.
OLTP partitionable OLTP skewed HTAP skewed HTAP partitionable
Workload Phase T h r o u g h p u t ( M t x / s ) OLTP Performance
DBx1000AnyDB
Figure 1: Performance of AnyDB across a workload evolvingfrom partitionable OLTP (phase 0-2), over a skewed OLTP(phase 3-5), to skewed HTAP (phase 6-8), and then to par-titionable HTAP (phase 9-11). The y-axis only shows thethroughput of the OLTP transactions excluding the OLAPqueries in the HTAP phases.
Contributions:
In this paper, we propose a radical new approachfor scale-out distributed DBMSs. Instead of hard-baking an archi-tectural model statically into the DBMS design, we aim for a newclass of so-called architecture-less DBMSs .The main idea of an architecture-less database system is that itis composed of a single generic type of component where multipleinstances of those components “act together” in an optimal manneron a per-query basis. To instrument generic components at run-time and coordinate the overall DBMS execution, each componentconsumes two streams: an event and a data stream. While the eventstream encodes the operations to be executed, the data stream shuf-fles the state required by these events to the executing component.Through this instrumentation of generic components by event anddata streams, a component can act as a query optimizer at onemoment for one query but for the next as a worker executing afilter or join operator.A key aspect of this execution model is that by simply chang-ing the event and data routing between generic components, anarchitecture-less DBMS can mimic different distributed architec-tures and form traditional architectures as well as completely newarchitectures. Another important aspect is, since we decide theevent and data routing on a per-query basis, an architecture-lessDBMS can simultaneously act as a shared-nothing system for onequery while acting as a shared-disk (disaggregated) DBMS for an-other query that runs concurrently with the first query. This opensup interesting opportunities for running mixed workloads, e.g.,HTAP, or adapting the execution to evolving workloads.Also, an interesting aspect of this execution model is that itcannot only mimic architectures on the macro-level (shared-nothingvs. shared-disk) but also can adapt execution strategies on the micro-level. For example, query execution in an architecture-less DBMScan mimic various query processing models at runtime (tuple-wisepipelined vs. vectorized vs. materialized) and degrees of parallelismby simply instrumenting the generic components with differentevent and data streams. The same holds also for other componentssuch as transaction execution and concurrency control. a r X i v : . [ c s . D B ] S e p iemo Bang, Norman May, Ilia Petrov, and Carsten Binnig Catalog+Stats (in) (b) AnyComponent / AC
Result (out)Table-Data (in)Scan-Op. (in)Query (in)
Server 2 (4 Cores)Server 1 (4 Cores)
DB 1 DB 2AC QOAC W ACW ACAC
ACQO E Worker E D
1. AC not acting2. AC acting as a Query Optimizer (QO)3. AC acting as Worker (Query Exec.)
Events (in)Data (in) D E ... E
Ops (out)Data (out)Events (out) D No Data (out)No Event (out) E D (a) AnyDB - Architecture-less DBMS Figure 2: AnyDB is an architecture-less DBMS of genericcomponents called AnyComponents (ACs), executing ar-bitrary logic. ACs are instrumented by events and datastreams. Depending on the incoming events an AC can actas a Query Optimizer (QO) or a Worker (W) executing a scanor a join operator or any other component, e.g., log writer,etc. An AC can also produce new data and event streams forother ACs. For example, an AC that acts as a scan operatorproduces a data stream with results of the scan operation.
The potential of the architecture-less database system is shownin Figure 1. Here, we compared the performance when running anevolving workload in a static shared-nothing architecture (blue line)based on an extended version of DBx1000 [9] compared to AnyDB(orange line) our prototypical implementation of an architecture-less database system. As we see, AnyDB is either able to meet theperformance of DBx1000 if its static architecture is optimal for theworkload or outperform DBx1000 where the static architecture isnot optimal, e.g., skewed workloads.
Outline:
The remainder of this paper is structured as follows. First,in Section 2 we give an overview of how we envision an architecture-less database system. Second, in Sections 3 and 4 we then discuss theopportunities that an architecture-less database provides for OLTP,OLAP as well as HTAP and present initial experimental resultsin each of these sections using our prototypical architecture-lessdatabase system AnyDB. Finally, we conclude with a discussion offuture directions in Section 5.
In the following, we first give an overview of the general executionmodel of an architecture-less DBMS. Afterwards we then presenthow typical database workloads can be mapped to this executionmodel and discuss the main challenges of this model.
As shown in Figure 2, the main idea of an architecture-less data-base system such as AnyDB is that the DBMS is composed only ofgeneric components, so-called
AnyComponents (ACs). These genericACs can provide any database functionality. By routing events andtheir required data to an AC, the AC can act as a query optimizerin one moment and in the next moment as a query executor or anyother component (e.g., log writer, etc.). This gives an architecture-less DBMS the flexibility to shift its architecture just in an instantwithout any downtime for reconfiguration, as we discuss later.To execute a complete SQL query (or a transaction composed ofa set of database operations), multiple events and data streams arerouted through AnyDB from one AC to another AC. For example, as outlined in Figure 2 (a), when executing a query in AnyDB, callingthe query optimizer is one event that can trigger follow-up eventsfor executing the operators, e.g., scans and joins. The accompanyingdata streams are responsible to shuffle the required state of an eventto the executing AC. Next, we focus on two important key designprinciples that underpin the architecture-less DBMS: • Fully Stateless / Active Data:
ACs are designed to be fullystateless meaning that events can be processed in any ACand all state (e.g., table data but also any other state suchas catalog data or statistics) required to execute an eventis being delivered to the AC via data streams. By makingACs fully stateless, we gain a high degree of freedom asany DBMS function can be executed anywhere. This featureallows mimicking diverse architectures but also to supportelasticity for all database functions individually, i.e., addi-tional ACs can execute any DBMS function at any time.Moreover, in architecture-less DBMSs data is active, mean-ing that data is not pulled after an event is scheduled but itis pushed from data sources actively to the ACs before it isactually needed (cf. Section 2.3). • Non-blocking / Asynchronous Execution:
A second keyaspect is that ACs are executing events in a non-blockingmanner. This means that an AC never waits for data of anevent if data is not available yet. Instead, another eventwhose data is available is being processed. For example, anoperator (e.g., a filter or a join) is only processed once itsinput data (e.g., a batch of tuples) has arrived via the datastream. To provide a non-blocking execution, ACs use queuesto buffer input events and data items. In addition, queuesdecouple the execution between ACs as much as possible;i.e., ACs can process events asynchronously from each other.This asynchronous execution model, which is only implic-itly synchronizing the execution across ACs through eventsand data streams, opens up many new opportunities, as wediscuss later in this paper.At a first glimpse, the execution model of an architecture-lessDBMS seems to have similarities with existing approaches suchas scalable stream processing systems or function-as-a-service of-ferings (aka server-less computing). However, there are crucialdifferences. (1) First and foremost, while AnyDB also uses streamsas a major abstraction, AnyDB is different from stream processingengines, since we target classical database workloads that processrelations and streams serve as a vehicle to on-the-fly adapt the data-base architecture on a per-query basis. Still, our approach benefitsfrom techniques of scalable stream processing such as efficient datarouting. (2) Similar to function-as-a-service, AnyDB relies on a fullystateless execution model. However, in architecture-less DBMSsdata (i.e., state) does not come as an afterthought. In classical server-less computing, a function is scheduled first and then data mustbe pulled in from storage before the execution can actually start.Instead, as mentioned before, in architecture-less DBMSs data isactive, meaning that data is actively pushed from data sources tothe ACs before the event is actually being processed. Moreover,while architecture-less DBMSs logically disaggregate the DBMSexecution into small functions like function-as-a-service, we stillallow executing events in a physically aggregated manner and alsoallow shipping events to the data to make use of locality. nyDB: An Architecture-less DBMS for Any Workload
In the following, we give a brief overview of how the executionmodel above can be used to execute OLAP and OLTP workloads.
Supporting OLAP:
The basic flow when executing an OLAP queryin an architecture-less DBMS is shown in Figure 3. The initial eventis typically a SQL query that is sent from a client to any AC of theDBMS – never mind which one – which then acts as the QueryOptimizer (QO). The main task of the QO is to come up with anefficient execution plan like a traditional query optimizer in a staticDBMS architecture.In contrast to traditional optimizers, however, the QO in anarchitecture-less DBMS produces an event stream and initiates thedata streams that instrument the ACs for query execution. Impor-tantly, the QO also determines the routing of a query’s events anddata through the architecture-less DBMS. Consequently, by theserouting decisions the QO defines the DBMS architecture perceivedby individual queries.For example, as shown in Figure 3 (a), if a query touches onlyone partition and there is moderate load in the system, then the QOcan route events of a query such that the architecture-less DBMSacts as a shared-nothing architecture. However, in case the queryload in the system increases, servers with additional ACs are addedand the architecture-less DBMS executes queries in a disaggregatedmode simply by routing events differently, as shown in Figure 3 (b).
Supporting OLTP:
The basic flow of executing OLTP transactionsis similar to executing OLAP read-only queries. Transactions arealso decomposed into event and data streams where routing deci-sions define the architecture. A key difference to read-only OLAP,however, is that in OLTP (1) transactions need to update stateand (2) concurrently running transactions need to coordinate theiroperations to guarantee correct isolation. Both these aspects arediscussed below in the following (cf.
Concurrency and Updates ). There are different key challenges to enable efficient executionin an architecture-less DBMS. One of them is the optimal routingof events and data for a given workload. Another one is to handleconcurrency and updates. In the following, we briefly discuss themain ideas how we aim to address these challenges. Some of theseideas are already built into our prototype DBMS AnyDB whileothers represent future routes of research.
Event and Data Routing:
As mentioned before, a key challenge of anarchitecture-less DBMS is to decide how to handle a query and howto route its events, as part of the query optimization. Depending onrequirements of an application (e.g., latency guarantees), load inthe system, and the workload, the query optimizer has to define anoptimal event routing. In our current prototype, we did not focus onthis problem but use an optimal decision to showcase the potentialof our approach. We believe however that this is an interestingavenue for learned query optimizers.A second challenging aspect is the efficient data routing. Asmentioned before, this aspect is important for latency hiding. Weutilize the decoupling of data streams from events in our executionmodel to solve this challenge. The main observation is that in DBMSexecution one often knows which data is accessed way ahead of timebefore the data is actually being processed. For example, complex
Legend:
Query Query Query Query
Warehouse1 Warehouse2
Join QOScan Scan JoinScan Scan
Warehouse 1
Scan Scan Scan ScanScan Scan Scan ScanJoin Join Join JoinQO QO QO QO QO
Warehouse 2 (a) AnyDB acting as Shared-Nothing (b) AnyDB acting as Shared-Disk
Query Query DataEventsQueryQuery
Figure 3: AnyDB can mimic diverse architectures simply byusing different routing schemes for events and data streams.In (a), two servers act as a shared-nothing database while in(b) additional resources (i.e., two servers with additional 4ACs per server) are added and AnyDB acts as a disaggregatedarchitecture to deal with a higher query load. For simplicity,we only show the events and data streams for (a). The gray-shaded boxes around the ACs, however, indicate in (b) whichACs execute events of the same query.
OLAP queries need to be optimized and compiled, often taking upto 100ms in commercial query optimizers in our experience, whilewe already know which tables contribute to a query before queryoptimization. In AnyDB we make use of this fact and initiate datastreams as early as possible. Once initiated a data stream activelypushes data to the AC where, for example, a filter operator will beexecuted once query optimization finished. We call this feature databeaming as data is often available at an AC before the accordingevent arrives, entirely hiding latencies of data transfers. We analyzethe opportunities of data beaming for OLAP later in Section 4.
Concurrency and Updates:
In general, updates are supported inAnyDB by event streams directed towards the storage which in-gests these events and produces acknowledgment events when theupdates have been processed, as required for transaction coordina-tion in OLTP. A major challenge in handling updates thus is to hidelatencies of updates as much as possible. Again, to hide latencies ofupdates and decrease the overall latency of executing a transaction,operations of one transaction are represented as events and exe-cuted asynchronously by ACs. For example, an update can be sent tothe storage by one AC while other (independent) operations of thetransaction can progress on other ACs. Only the commit operationat the end of a transaction needs to know if the write successfullypersisted and thus needs to wait for the acknowledgment eventcoming from the storage. As we show later in Section 3, this asyn-chronous model for OLTP provides many interesting opportunitiesand results in higher performance under various workloads.Another challenge that is harder to solve is to efficiently handleconcurrency. A naïve way would be to implement a lock managerusing events representing lock operations and data streams provid-ing the state of the lock table. A more clever way, however, is torethink concurrency protocols and route events and data streamssuch that their processing order already captures the needs of aparticular isolation level for concurrency control, as we discusslater in Section 3. iemo Bang, Norman May, Ilia Petrov, and Carsten Binnig
E............ E............KeyKey E xc l u s i v e + QORIDIndex.lookup(...)RIDLock.acquire(...)RecordRecord.read(...) R I D Record.update(...)Lock.release(...) ...
Index.lookup(...) (a) QO PaymentTxn QO PaymentTxnScanCust.E ED QO InsertHist.UpdateWHE ED UpdateDist.D EUpdateCust.E (b) (c) (d)
E............E............
Figure 4: Duality of Disaggregation. (a) shows how a trans-action is logically disaggregated into individual events foreach operation. (b) shows physically aggregated executionof events by routing the stream to one AC. (c) shows eventrouting for fully intra-transaction parallel execution. (d)shows precise intra-transaction parallel execution.
Fault-Tolerance and Recovery:
Fault-tolerance and recovery are twomajor challenges any DBMS needs to address. For an architecture-less DBMS this is a challenge due to the asynchronous (decoupled)execution of multiple ACs where individual ACs might fail.Again, a naïve approach would be to implement standard write-ahead logging by sending log events from ACs to durable storage.For recovery, the DBMS could be stopped and the log could be usedto bring the DBMS into a correct state. Again, in an architecture-less DBMS we believe that we can do better and learn from thestreaming community. For example, as the entire execution of aDBMS is represented as streams, another direction is to make thestreams reliable, such that upon AC failure the streams (events anddata) can be rerouted to another AC. Applying these ideas is againan interesting avenue of future research.
In the following, we discuss the various opportunities emergingfrom an architecture-less DBMS when executing OLTP workloadsand show initial results when compared to existing execution mod-els of static architectures (shared-nothing and shared-disk). For allinitial experiments in this paper, we use the two dominant transac-tions of the TPC-C benchmark (i.e., payment and new-order).
As indicated earlier in Figure 1, for partitionable OLTP workloadsan architecture-less DBMS can achieve nearly the same through-put as an (aggregated) shared-nothing architecture. Key to this isthe duality of disaggregation in the architecture-less DBMS. Thearchitecture-less DBMS distinguishes logical disaggregation of theDBMS design and physical disaggregation of the DBMS execution.Logically, the DBMS is entirely disaggregated into independentfine-grained functionality interacting via events and data streams.However, while the overall execution is disaggregated into manysmall events, physically the events can still be executed in an aggre-gated manner if desired. This opens up the opportunity to achievehigh data locality if desired as all events can be executed close tothe data, e.g., a partition of the database. For example, an OLTP transaction may consist of an event streamlike in Figure 4 (a). Yet, this logical disaggregation does not mandatedisaggregated execution. In the contrary, any sub-sequence of theseevents can be physically aggregated and executed by a single AC.As shown in Figure 4 (b), the entire event stream of a transactioncan be executed by a single AC. In fact, this physical aggregationof events establishes a shared-nothing architecture that performson par with the static shared-nothing architecture of DBx1000 asshown in Figure 1.
The Gist:
Through this duality of disaggregation (logical vs.physical), we believe that an architecture-less DBMS can efficientlymimic diverse architectures ranging from entirely aggregated shared-nothing to fine-grained disaggregated as needed, simply shiftingbetween those by adapting event and data routes.
Along with the freedom of achieving different architectures onthe macro-level, the execution model in an architecture-less DBMSalso provides broad freedom on the micro-level to layout parallelexecution strategies in an optimal manner.Generally, as explained earlier, transactions are represented asevent streams flowing through the architecture-less DBMS. Impor-tantly for transaction execution, this event-based execution allowsdiverging from typical execution models in OLTP that aim for inter -transaction parallel execution and allows investigating also otherforms of parallelism. For example, event-based execution naturallybrings opportunities to ad hoc parallelize execution within a sin-gle transaction to achieve intra -transaction parallelism, especiallywhen contention prohibits inter-transaction parallel execution.The efficiency of this freedom to change the transaction exe-cution on the micro-level becomes also visible in Figure 5. Whenrunning a partitionable OLTP workload in the first phase (0-2),AnyDB mimics not only a shared-nothing architecture but alsouses classical inter-transaction parallelism. Afterwards it uses intra-transaction parallelism in the second phase (3-5) for the highlyskewed, contended OLTP workload, where 100% of TPC-C pay-ment transactions operate on one warehouse only.The baseline DBx1000 in this experiment uses a shared-nothingmodel and is thus bound by the resources that are assigned to onepartition resulting in 0.7 M txn/s. In such a case, our architecture-less DBMS allows to simply shift from inter- to intra-transactionparallelism by routing events of a single transaction to severalACs, i.e., shifting from Figure 4 (b) to (c). This intra-transactionparallel execution may accelerate transactions in contended OLTPworkload, as already proven by architectures such as DORA [6].However, just like any design decisions in a static architecture,also static intra-transaction parallelization does not always provebeneficial either. For example, Figure 5 shows that with naïve intra-transaction parallelism, where every independent operation of atransaction is farmed out to a different AC, AnyDB only achieves0.8 M txn/s (orange squares), barely exceeding the inter-transactionparallel execution of the DBx1000 baselines (blue lines). The mainreason is that the overhead of parallelizing within one transactiondominates the execution.The architecture-less DBMS addresses the challenging paral-lelization of transactions in the following ways: the representationof transactions as event streams allows the architecture-less DBMS nyDB: An Architecture-less DBMS for Any Workload
OLTP partitionable OLTP skewed
Workload Phase T h r o u g h p u t ( M t x / s ) OLTP Performance
DBx1000 4TEDBx1000 1TEAnyDB Shared-NothingAnyDB Streaming CCAnyDB Static Intra-TxnAnyDB Precise Intra-Txn
Figure 5: OLTP performance of AnyDB versus the shared-nothing DBx1000 under partitionable and skewed OLTP.In phases 0-2 AnyDB acts as a shared-nothing DBMS us-ing an inter-transaction parallel execution model while inphases 3-5 AnyDB acts as a shared-disk DBMS using anintra-transaction parallel execution model. Note that forDBx1000, 4 transaction executors (TEs) perform like a sin-gle TE due to high contention between transactions. to route independent sub-sequences of events (i.e., sub-sequences ofoperations) to individual ACs for parallel execution. Thus, the mainchallenge in an architecture-less DBMS is to split a transaction intosuitable sub-sequences of events and route them to different ACs,balancing the amount of work versus overhead. In our experiments,for example, we partition the TPC-C payment transaction into onesub-sequence with several brief update statements and a secondsub-sequence with a long range scan, as depicted in Figure 4 (d).Finding an optimal splitting and routing of event sequencesof transactions depends on many factors. One important point isthat the individual sub-sequences of operations have similar exe-cution latencies, such that the overall latency of the transaction isminimized. Finding such an optimal routing of events is an opportu-nity for learning query optimization and scheduling, as mentionedbefore. As shown in Figure 5, with this balanced (i.e., precise) intra-transaction parallelization AnyDB achieves 1.2 M txn/s (orangetriangle) with only 2 ACs, outperforming the baseline (blue lines)and AnyDB’s naïve parallelization (orange square) by 3.2x and 3xin throughput per thread, respectively.
The Gist:
Generally, we envision, that the freedom of the ex-ecution models on the micro-level in an architecture-less DBMScan enable new parallel transaction execution models spanningany design between pure inter- to aggressive (fine-grained) intra-transaction parallelism on a per-transaction level.
In OLTP workloads, especially under high contention, concur-rency control (CC) causes significant coordination effort and chal-lenges efficient parallelization. In an architecture-less DBMS, theevent-based nature of transactions provides the opportunity totransform CC to a streaming problem. Thereby, the architecture-less DBMS can improve the efficiency of traditional CC schemesand opens many opportunities for CC schemes, as discussed next.
Transforming Traditional Concurrency Control:
Interestingly, manytraditional CC protocols are stream-like already and thus benefitfrom a direct mapping to the asynchronous (non-blocking) exe-cution model. For example, a pessimistic lock-based CC schemeneeds to match incoming lock requests with its lock state. Thiscan be mapped to a streaming join on an event stream containing lock requests and a data stream containing the lock state of therequested item. Similarly, verification in optimistic CC protocolsjoins the read/write set of a transaction which is one data streamwith the current state of the database which is another data stream.Yet, an architecture-less DBMS also offers opportunities for novelcoordination-free CC schemes vastly outperforming these tradi-tional approaches, to be explained in the following.
Novel Streaming Concurrency Control:
The key idea of rethink-ing CC schemes is that they can be enabled by efficiently orderingevents of (conflicting) transactions flowing through the architecture-less DBMS, rather than actively synchronizing execution of con-current transaction using traditional CC schemes. Especially underhigh contention, traditional CC causes high coordination overheads.Here, the streaming execution of transactions brings new opportu-nities despite high contention.Concurrency control in AnyDB can be implicitly and coordination-free encoded into event routes. That is, for consistency of concur-rent transactions it suffices to route their events in a consistentorder through ACs which execute the conflicting operations. Forexample, considering two TPC-C payment transactions accessingthe same warehouse, AnyDB can guarantee consistency by simplyrouting their events to all involved ACs in the same order. Thereby,AnyDB enables intra-transaction parallelism, routing independentevents through different ACs and also provides CC without theneed to actively synchronize operations at the same time.Note that event-ordering does not violate our non-blocking(asynchronous) execution model of ACs, since operations (i.e., events)of conflicting transactions simply remain in the ACs’ input queuesfor ordering while other events can still be executed. In Figure 5,we see that this instantiation of AnyDB called streaming CC (or-ange pentagon), yields 1.7 M txn/s for TPC-C payment under highcontention. This is much closer to the performance of the uniform(partitionable) execution of TPC-C payment in phases 0-2.
The Gist:
Along the properties of event streams, we envisionnovel CC protocols, that avoid active synchronization, as discussed.Moreover, the streaming CC enables new directions where eventsare gradually rerouted depending on the load, e.g., at first all eventsfor a specific transaction are routed to a single AC until this ACbecomes overloaded. Then AnyDB may transparently repartitionevent streams while still keeping consistent event order.
Previously, we have described that the execution model of anarchitecture-less DBMS provides many opportunities for OLTP.In the following, we discuss further opportunities for OLAP as wellas mixed HTAP workloads.Especially for OLAP, operations encoded in the event streamsare data intensive (e.g., a join of two large tables). Therefore, datastreams must efficiently bring data to wherever events are executedas if data access was local. While this aspiration of “omnipresent”data appears challenging, we observe that in DBMSs one oftenknows data to be accessed way ahead of time before actually pro-cessing it. For example, in OLAP data is only accessed and processedafter several milliseconds of query optimization. Hence, we propose data beaming , a technique initiating data streams early and pushingdata to ACs where events will be executed. iemo Bang, Norman May, Ilia Petrov, and Carsten Binnig E x e c u t i o n T i m e ( m s ) (a) Query 0 10 20 30 400510Query Compile Time (ms)(b) Build 0 10 20 30 4001020 (c) Probe BaselineBeam BuildBeam Build & ProbeAggregatedDisaggregatedCompile Time DB-C
Figure 6: Data beaming can effectively shorten query execu-tion for disaggregated execution of OLAP workloads and inthe best case hide latencies of data shuffling completely.
In the following experiment, we demonstrate the effect of databeaming with a simple OLAP query. Based on CH-benCHmarkQ3 [3], our query reports all open orders for all customers fromstates beginning with “A” since 2007 via 3 (filtered) scans and 2 joins.In Figure 6, we see the effect of data beaming in several degreesfor this query: (1) In blue, the baseline does not utilize beaming,but instead passively pulls in data when needed. (2) In orange, thebuild sides are beamed during query compilation, then joins areexecuted. (3) In green, build and probe sides are beamed.To detail the efficiency of data beams, we implement two vari-ants: one where data beams only need to shuffle locally (not over thenetwork) and one where we shuffle data across the network. Thesolid lines in Figure 6 demonstrate the runtime using local beamingvia shared-memory queues (e.g., to hide NUMA latencies) andthe dashed lines show the beaming across the network for a dis-aggregated architecture using DPI-flows [1]. On the x-axis, 30msmarks the time taken by a commercial DBMS (DB-C) to compilethis query.Figure 6 (a) shows that the overall query execution time withbeaming is only slightly higher than query compile time (greenline), whereas without beaming (blue line) the query executiontime has additional latency of 20ms, since data transfer is not over-lapped with query compile time. In detail, Figures 6 (b) and (c) showthe individual effects of data beaming on the build and probe side(without query compilation overhead), respectively. We see thatbeaming can reduce the runtime of the smaller build side almost to0ms. For the larger probe side, beaming also reduces the runtimefrom 30ms to less than 10ms. Notably, the disaggregated architec-ture (which needs to shuffle data across the network) performs evenbetter than the aggregated architecture, as DPI offloads event anddata transfers to the InfiniBand NICs and acts as a co-processor inAnyDB’s processing model.The previous experiment demonstrated the utility of data beam-ing to hide data transfer latencies in OLAP workloads. Besideshiding transfer latencies, data beaming can also be used to achieveother goals such as resource isolation, e.g., for HTAP workloads.The idea is that in HTAP workloads, we can use data beams to routedata intensive analytical queries to additional compute resourcesdisaggregated from storage while latency-sensitive transactions areexecuted close to the data.The HTAP workloads in Figure 1 (phase 6-11) outline such sce-narios, where the OLAP query of the previous experiment is exe-cuted in parallel to the OLTP workload. Here, AnyDB executes theOLAP query independently of the OLTP workload, only sharing Github: Folly single-producer-single-consumer queue storage resources, whereas the OLAP query in the static DBx1000uses the same transaction resources for OLAP queries as for theOLTP workload. Thereby, AnyDB simultaneously provides higherOLTP and OLAP performance than DBx1000.
The Gist:
In general, we envision AnyDB to establish flexiblearchitectures through described data beaming as well as optimal perquery/transaction event routing, opening up new paths to hybridarchitectures and supporting various types of deployments.
In this paper, we have proposed architecture-less DBMSs, a radicalnew approach for scale-out distributed DBMSs. In addition to thediscussed opportunities, we see many further interesting researchdirections arising from the architecture-less approach:
Elasticity for Free:
Flexible event routing and data beaming openup opportunities, apart from resource isolation, for transparentelasticity without additional latencies. Considering events are self-contained and state is always beamed, elasticity for execution ofevent streams just means consistent routing of events and theirstate to an elastic number of ACs. Even more, as mentioned beforesince all events and data are both delivered as streams to ACs, thesestreams could be repartitioned or rerouted to distribute load in thesystem adaptively.
Transparent Heterogeneity:
The stateless execution model in con-junction with the opportunity for elasticity further facilitates trans-parent (ad hoc) integration of heterogeneous compute resourcesper query, including but not limited to accelerators (e.g., FPGAs orGPUs) and programmable data planes (e.g., programmable NICs orswitches). Moreover, since events fully describe what to do and datastreams deliver all required state, event execution ( how to do it ) canbe specialized for FPGAs, etc. without any impact nor dependencieson the rest of the DBMS.
Crossing Clouds and More:
Finally, data beaming is an interestingconcept generally hiding data transfer cost not only within butalso across data centers. This opens up opportunities for DBMSdeployments across on-premise, cloud offerings, and the edge with-out paying significant latencies for data transfer. For example, anarchitecture-less DBMS for HTAP workload may run transactionsfor daily business on-premise and may ad hoc beam data to cloudresources for sporadic reporting, combining the benefits of bothplatforms efficiently.
References [1] G. Alonso et al. DPI: the data processing interface for modern networks. In
CIDR ,2019.[2] T. Bang et al. Robust performance of main memory data structures by configura-tion. In
SIGMOD’20 , pages 1651–1666, 2020.[3] R. Cole et al. The mixed workload ch-benchmark. In
Proceedings of the FourthInternational Workshop on Testing Database Systems , page Article 8. ACM, 2011.[4] A. K. Goel et al. Towards scalable real-time analytics: an architecture for scale-outof olxp workloads.
Proc. VLDB Endow. , 8(12):1716–1727, 2015.[5] A. Gupta et al. Amazon Redshift and the case for simpler data warehouses. In
SIGMOD’15 , page 1917–1923. ACM, 2015.[6] I. Pandis et al. Data-oriented transaction execution.
Proc. VLDB Endow. , 3(1), 2010.[7] R. Taft et al. E-store: Fine-grained elastic partitioning for distributed transactionprocessing.
Proc. VLDB Endow. , 8(3):245–256, 2014.[8] A. Verbitski et al. Amazon Aurora: Design considerations for high throughputcloud-native relational databases. In
SIGMOD’17 , page 1041–1052. ACM, 2017.[9] X. Yu et al. Staring into the abyss: An evaluation of concurrency control with onethousand cores.