Real-time indexing database vendor Rockset has advanced its vision of a converged database with the general availability of new integrations with MySQL and PostgreSQL databases.
Rockset, based in San Mateo, Calif., Has developed database technology that enables analytical queries on different types of data, including time series, structured and unstructured data.
A key challenge, however, has been with operational data stores where data is stored in a relational database such as MySQL or PostgreSQL as a record system. Rockset aims to overcome this problem with direct integration that makes it easy to replicate data from MySQL and PostgreSQL compatible databases into Rockset, where users can then use the data for real-time analysis.
Rockset made the new integrations generally available on July 8. The MySQL and PostgreSQL integration connectors have been available as preview technology since April and have had a few early adopters.
Igor Blumberg, chief technology officer at Dimona, said the company uses Amazonian Aurora, a MySQL compatible cloud database. Blumberg noted that Dimona’s database system monitors and maintains data on all aspects of the company’s operations, including finance, inventory, production and sales.
Rockset for inventory tracking
One problem that Dimona has faced is how to track her inventory accurately and in real time.
Blumberg said the company has multiple warehouses. In each warehouse there are several locations for inventory, such as shelves or pallets with clothes. Blumberg called these locations addresses that constantly change over the course of a day. Meanwhile, Dimona needs to know where all of her inventory is.
Rockset provides a set of data connectors that help extract data from a source destination so that it can be indexed into Rockset’s real-time database.
To help enable real-time data visibility, Dimona replicates the Aurora database on Rockset, then queries Rockset for the current inventory. Blumberg said that the live replication from MySQL to Rockset made it easy to keep all data up to date.
“We don’t have to worry about updating the database cache layer,” Blumberg said. “We can focus on our business and let Rockset do the inventory work.”
How Rockset MySQL and PostgreSQL Integration Works
Co-founder and CEO of Rockset Venkat Venkataramani explained that users had previously also been able to extract data from MySQL and PostgreSQL databases.
We don’t have to worry about updating the database cache layer. We can focus on our business and let Rockset calculate the inventory.
Igor blumbergTechnology Director, Dimona
Venkataramani said the difference between the new MySQL and PostgreSQL integration and what users were doing before is that the previous process was more manual and slower. Rockset users could extract data in batch mode at periodic intervals, such as hourly or daily. On the other hand, the new integration allows data replication in real time.
Rockset first scans and copies all data from an existing MySQL or PostgreSQL database. After this initial copy of the data, Rockset moves on to what Venkataramani called a “quick follow” model.
This model provides real-time data replication using edit data capture (CDC) which are already present in MySQL and PostgreSQL. CDC provides a feed of all new insertions, updates, and deletions of data to existing records in a database.
Venkataramani explained that as data arrives from the source MySQL or PostgreSQL database through CDC, it is automatically indexed in Rockset and turned into quick SQL query tables.
By allowing users to extract data from online transaction processing (OLTP) such as MySQL and PostgreSQL, Venkataramani said that users can now also benefit from online analytics processing (OLAP) the capabilities of Rockset’s real-time indexing database.
Venkataramani said the vendor’s roadmap is to continue to develop integrations and connectors for data that can be pulled from the Rockset database.
“No matter where your data is, we want to create a real-time connector to that dataset,” he said.
