Tirthankar Lahiri

Fast-Start: quick fault recovery in oracle

Availability requirements for database systems are more stringent than ever before with the widespread use of databases as the foundation for ebusiness. This paper highlights Fast-Start#8482; Fault Recovery, an important availability feature in Oracle, designed to expedite recovery from unplanned outages. Fast-Start allows the administrator to configure a running system to impose predictable bounds on the time required for crash recovery. For instance, fast-start allows fine-grained control over the duration of the roll-forward phase of crash recovery by adaptively varying the rate of checkpointing with minimal impact on online performance. Persistent transaction locking in Oracle allows normal online processing to be resumed while the rollback phase of recovery is still in progress, and fast-start allows quick and transparent rollback of changes made by uncommitted transactions prior to a crash.

Oracle Database In-Memory: A dual format in-memory database

The Oracle Database In-Memory Option allows Oracle to function as the industry-first dual-format in-memory database. Row formats are ideal for OLTP workloads which typically use indexes to limit their data access to a small set of rows, while column formats are better suited for Analytic operations which typically examine a small number of columns from a large number of rows. Since no single data format is ideal for all types of workloads, our approach was to allow data to be simultaneously maintained in both formats with strict transactional consistency between them.

Distributed architecture of Oracle database in-memory

Over the last few years, the information technology industry has witnessed revolutions in multiple dimensions. Increasing ubiquitous sources of data have posed two connected challenges to data management solutions -- processing unprecedented volumes of data, and providing ad-hoc real-time analysis in mainstream production data stores without compromising regular transactional workload performance. In parallel, computer hardware systems are scaling out elastically, scaling up in the number of processors and cores, and increasing main memory capacity extensively. The data processing challenges combined with the rapid advancement of hardware systems has necessitated the evolution of a new breed of main-memory databases optimized for mixed OLTAP environments and designed to scale. The Oracle RDBMS In-memory Option (DBIM) is an industry-first distributed dual format architecture that allows a database object to be stored in columnar format in main memory highly optimized to break performance barriers in analytic query workloads, simultaneously maintaining transactional consistency with the corresponding OLTP optimized row-major format persisted in storage and accessed through database buffer cache. In this paper, we present the distributed, highly-available, and fault-tolerant architecture of the Oracle DBIM that enables the RDBMS to transparently scale out in a database cluster, both in terms of memory capacity and query processing throughput. We believe that the architecture is unique among all mainstream in-memory databases. It allows complete application-transparent, extremely scalable and automated distribution of Oracle RDBMS objects in-memory across a cluster, as well as across multiple NUMA nodes within a single server. It seamlessly provides distribution awareness to the Oracle SQL execution framework through affinitized fault-tolerant parallel execution within and across servers without explicit optimizer plan changes or query rewrites.

50,000 users on an Oracle8 universal server database

In this paper, we describe the Oracle Large User Population Demonstration and highlight the scalability mechanisms in the Oracle8 Universal Data Server which make it possible to support as many as 50,000 concurrent users on a single Oracle8 database without any middle-tier TP-monitor software. Supporting such large user populations requires many mechanisms for high concurrency and throughput. Algorithms in all areas of the server ranging from process and buffer management to SQL compilation and execution must be designed to be highly scalable. Efficient resource sharing mechanisms are required to prevent server-side resource requirements from growing unboundedly with the number of users. Parallel execution across multiple systems is necessary to allow user-population and throughput to scale beyond the restrictions of a single system. In addition to scalability, mechanisms for high availability, ease-of-use, and rich functionality are necessary for supporting complex user applications typical of realistic workloads. All mechanisms must be portable to a wide variety of installations ranging from desk-top systems to large scale enterprise servers and to a wide variety of operating systems.

Accelerating analytics with dynamic in-memory expressions

Oracle Database In-Memory (DBIM) accelerates analytic workload performance by orders of magnitude through an in-memory columnar format utilizing techniques such as SIMD vector processing, in-memory storage indexes, and optimized predicate evaluation and aggregation. With Oracle Database 12.2, Database In-Memory is further enhanced to accelerate analytic processing through a novel lightweight mechanism known as Dynamic In-Memory Expressions (DIMEs). The DIME mechanism automatically detects frequently occurring expressions in a query workload, and then creates highly optimized, transactionally consistent, in-memory columnar representations of these expression results. At runtime, queries can directly access these DIMEs, thus avoiding costly expression evaluations. Furthermore, all the optimizations introduced in DBIM can apply directly to DIMEs. Since DIMEs are purely in-memory structures, no changes are required to the underlying tables. We show that DIMEs can reduce query elapsed times by several orders of magnitude without the need for costly pre-computed structures such as computed columns or materialized views or cubes.

Fault-tolerant real-time analytics with distributed Oracle Database In-memory

Modern data management systems are required to address new breeds of OLTAP applications. These applications demand real time analytical insights over massive data volumes not only on dedicated data warehouses but also on live mainstream production environments where data gets continuously ingested and modified. Oracle introduced the Database In-memory Option (DBIM) in 2014 as a unique dual row and column format architecture aimed to address the emerging space of mixed OLTAP applications along with traditional OLAP workloads. The architecture allows both the row format and the column format to be maintained simultaneously with strict transactional consistency. While the row format is persisted in underlying storage, the column format is maintained purely in-memory without incurring additional logging overheads in OLTP. Maintenance of columnar data purely in memory creates the need for distributed data management architectures. Performance of analytics incurs severe regressions in single server architectures during server failures as it takes non-trivial time to recover and rebuild terabytes of in-memory columnar format. A distributed and distribution aware architecture therefore becomes necessary to provide real time high availability of the columnar format for glitch-free in-memory analytic query execution across server failures and additions, besides providing scale out of capacity and compute to address real time throughput requirements over large volumes of in-memory data. In this paper, we will present the high availability aspects of the distributed architecture of Oracle DBIM that includes extremely scaled out application transparent column format duplication mechanism, distributed query execution on duplicated in-memory columnar format, and several scenarios of fault tolerant analytic query execution across the in-memory column format at various stages of redistribution of columnar data during cluster topology changes.

Oracle TimesTen Scaleout: A New Scale-Out In-Memory Database Architecture for Extreme OLTP

Oracle TimesTen Scaleout is a shared-nothing scale-out inmemory database designed for extreme OLTP workloads, such as IoT, real-time fraud detection, telecommunications etc. TimesTen Scaleout features rich SQL including complex queries with joins, aggregations and analytic functions, transparent distributed execution, full ACID multi-statement transactions, global secondary indexes and sequences. The design features built-in high availability using a k-safe data duplication mechanism, and transparent failure handling in order to minimize application down time. All management functions such as installation, configuration, and monitoring are provided via a centralized management repository. We describe some of the challenges in developing such a scale-out in-memory database architecture and show extreme performance results exceeding 100 million transactions per second and 1 billion SQL selects per second.

How does Oracle Database In-Memory scale out?

The Oracle RDBMS In-memory Option (DBIM), introduced in 2014, is an industry-first distributed dual format in-memory RDBMS that allows a database object to be stored in columnar format purely in-memory, simultaneously maintaining transactional consistency with the corresponding row-major format persisted in storage and accessed through in-memory database buffer cache. The in-memory columnar format is highly optimized to break performance barriers in analytic query workloads while the row format is most suitable for OLTP workloads. In this paper, we present the distributed architecture of the Oracle Database In-memory Option that enables the in-memory RDBMS to transparently scale out across a set of Oracle database server instances in an Oracle RAC cluster, both in terms of memory capacity and query processing throughput. The architecture allows complete application-transparent, extremely scalable and automated in-memory distribution of Oracle RDBMS objects across multiple instances in a cluster. It seamlessly provides distribution awareness to the Oracle SQL execution framework, ensuring completely local memory scans through affinitized fault-tolerant parallel execution within and across servers without explicit optimizer plan changes or query rewrites.