In part 1 we looked at Netezza’s TwinFin line and its challenges. In part 2 we’ll look at a similar case of “what’s new is old” -- Oracle’s Exadata v2 announcement.

This was just announced with great fanfare in typical Larry Ellison style. Key details include:
- The hardware is now Sun rather than HP (as you might expect given Oracle’s intention to acquire Sun)
- Each rack has 5TB of flash storage, with lofty claims about the performance benefits
- ‘Hybrid columnar compression’ which is simply a new compression technique (and not a column-store technology)
- Pricing is $1.15M per rack -- $20k/TB for SAS (600GB drives) / $5.7k/TB for SATA (2TB drives) -- PLUS $1.68M/rack for Oracle’s database software
- Claims of I/O of 21GB/s per rack, and that the overall performance is 2x the original Exadata

Lots of fanfare, and to their credit they are running on some nice hardware. But Exadata v1 ran on nice HP hardware, and yet it was anything but a commercial success. So what is the elephant in the room that makes customers so wary of this Oracle offering? And does v2 do anything to address these issues?

What is the benefit of Exadata compared to prior Oracle offerings? Exadata eliminates the SAN bottleneck. i.e. Previously all Oracle RAC nodes needed to share the the bandwidth from a single SAN, which led to very slow I/O rates that wouldn’t scale with additional RAC nodes. Exadata fixes this by providing a scalable storage layer that replaces the role of SAN.

However, this only addresses one of the smaller issues with Oracle’s scaling story. The bigger issue is that Exadata relies on RAC for its scaling, and RAC is complex, has scaling problems and is unstable — and wasn’t designed for the kind of large parallel analytics queries seen in DW workloads. Anyone who has tried to build a 10TB+ warehouse with Oracle technology has likely felt the brunt of these limitations — Exadata or not. And nothing in Exadata v1 or v2 does anything to change this.

RAC & Parallel Query have Scaling Problems

Unlike all the leading data warehouse database vendors (Teradata, Greenplum, Netezza), RAC (with or without Exadata) isn’t a shared-nothing MPP architecture. It requires that all nodes can see all disk, and utilizes locking to mediate access. Users will often see poor scaling and slow loading due to locking contention and inefficiencies. Data motion between nodes is very inefficient compared to MPP shared-nothing architectures that have unencumbered node-to-node communication, and RAC will often broadcast large amounts of data between nodes unnecessarily and cause overall slowness and blocking of query processing.

RAC & Parallel Query are complex to configure and work unpredictably

Parallel query is complex to configure and control. There are many cases where it will produce very poor plans (lots of broadcast, switching to poor nested loop or sort-merge plans, or other large inefficiencies despite having good statistics and partitioning) or forgo any use of parallelism (e.g. running out of parallel query slaves). Setting up storage partitioning (aka distribution) requires explicit placement of data during loading — unlike MPP shared-nothing architectures where data being loaded is automatically distributed between nodes based on the distribution key of the table.

RAC is Unstable

Due to its complexity, there are many issues and instabilities that remain despite years of development. We have many first-hand stories of RAC instability, scaling issues, and bugs.

The bottom line is that Oracle is a fine mainstream OLTP database. However, the very architectural decisions that allowed it to succeed in that market are why it has struggled so badly for so many years at high scale and highly parallel workloads. Lacking a true MPP shared-nothing architecture, customers quickly learn that the system requires an endless amount of ‘black magic’ tuning and partitioning to keep things from unraveling. This is a world apart from Greenplum, where, for example, we were able to deploy a 6.5 Petabyte database across 96 nodes at eBay that supports a wide class of users and queries with little fanfare or tuning required.

Note: For a still very relevant review of the Exadata v1 announcement, see my earlier blog post: What is Oracle Exadata?