Crate.io Releases Commercial Machine Data Platform

Crate.io on Tuesday introduced a commercial Machine Data Platform, along with a new version of its open source SQL database for the Internet of Things and machine data. The company also announced an US$11 M Series A funding round.

The Machine Data Platform is Crate.io's first major commercial venture following last year's initial steps toward selling services around its free database, noted Andy Ellicott, chief product and marketing officer at the company.

Crate.io has reached several critical fiscal and technical milestones since then, which reflect its steadfast commitment to meeting the needs of users as they engage with the next generation of machine data, said CEO Christian Lutz.

Hosted on Microsoft Azure, the new Crate Machine Data Platform enables enterprises to act on the insights derived from the data CrateDB collects. The platform is powered by CrateDB and is an environment for putting IoT data to work.


What Is CrateDB?
CrateDB 3.0 is an ASCII SQL database designed for real-time processing. It is beneficial for managing and monitoring sensor data, logs, messages, and device-generated GPS data. CrateDB 3.0 is particularly useful to people who have very large sensor-based data tables or a large number of unique values like customers.

The upgrade offers faster performance and enhanced security. It gives mainstream SQL developers access to machine data applications that previously were available only with NoSQL solutions.

The database is designed to scale query throughput linearly with cluster size, making capacity both more predictable and more affordable than other solutions.

"The machine data platform is really a data layer that makes the data management part of these IoT platforms a turnkey process," Crate.io's Ellicott told LinuxInsider.

Turning Heads
The driving factor for CrateDB is providing a great customer service experience for users. It comes down to the ability to do data machine things in SQL, according to Ellicott.

"There is still a lot of ignorance about the limitations of NoSQL databases. Crate takes the risk and mystery out of that to make it a sure bet," he said.

Traditionally, CrateDB competes against outfits like Splunk and Elastic. Crate is traditionally an SQL alternative, Ellicott noted.

"That means nearly everybody in IT and software development knows how to use CrateDB. The number of SQL developers far outnumbers the people who have skills in the other tools, he explained. "Crate is an easier fit, bringing the machine data to mainstream SQL developers. It also makes Crate more competitive."

Crate.io faces a very tough competitive challenge, suggested Roman Shaposhnik, vice president of product and strategy at Zededa.

CrateDB began more as a document store than a database, he pointed out, but it quickly added SQL capabilities -- in part by integrating the Presto SQL query engine from Facebook.

"By doing so, it entered the competition with more established Massively Parallel Processing database vendors, including open source ones like Greenplum," Shaposhnik told LinuxInsider. "Within the MPP market, CrateDB seems to be focusing on machine data uses cases more than anything else, but the jury is still somewhat out on how much better they will prove to be there."

Open Source Upgrade
The major improvement in this latest database release is the implementation of the new hash join function, according to Ellicott.

A speed boost enables CrateDB 3.0 to run 20 to 100 times faster. A new Views feature allows users to create custom virtual data tables, with each table having different security settings.

Among CrateDB 3.0's other enhancements:

enhanced security features, including user administration and role-based access control;
hyperlog log, an algorithm used to get aggregations to generate answers orders of magnitudes faster while still providing estimates;
sub-queries with Update and Delete statements;
a Shard Visualizer to make database administration easier.
The market for these types of applications is vast, Ellicott said. The database helps to predict and monitor the behavior of IoT devices. Increasingly customers are using that data to control the very thing they are monitoring.

Use Cases
In manufacturing, managers use it to improve the factory operations in areas such as quality control and predictive maintenance on the machinery.

Other uses involve improving safety and warehouse logistics. The data helps to improve fleet management through location tracking, for example.

"We see machine data being put to work on e-commerce sites, as well as retail operations via RFID and video analytics," said Ellicott. "Most customers use the database to build IoT platforms for internal use or for commercial purposes. They are using the database and the data to take data generated by a thing, and analyzing it using complex methods based on AI and statistical algorithms."

Crate Machine Data Platform
The Crate Machine Data Platform resulted from the company's experience working with machine data innovators. The data platform helps companies put machine data to work faster by simplifying the time-consuming job of building and operating the data management plumbing that forms the foundation for IoT systems.

Progressive enterprises and IoT startups have been racing to build new systems that harness data generated by Internet-connected factories, vehicles and smart buildings so they can put it to work immediately. These systems ingest streams of sensor data, analyze it and then trigger alerts or actions in order to optimize the behavior of IoT devices, according to Crate.io.

The platform lets users change factory machine settings to improve output, to direct failing fleet vehicles off the road to increase safety, or to disable access credentials to reduce fraud. It is a cloud-native stack available as a hosted offering on Microsoft Azure, or as an on-premises solution.

"We are taking the risk, the time to market, and the cost for these companies to build a platform off the table," said Ellicott, "allowing the companies to focus on what makes their solution valuable to the customer, and bring them to production or market much faster."

Core Features
The platform is powered by CrateDB. Every customer has a container environment plus a feature for data loading from sensors or a message queuing system like Kafka or RabbitMQ, noted Ellicott.

Crate.io will operate it and monitor it 24 X 7 for customers for a monthly subscription fee. The platform comes in various sizes, starting at $700 per node per month.

Key features of the new platform:

High-Velocity Data loading - scalable service ingests millions of SQL, JSON, MQTT data points per minute into the platform ... any data structure, even complex JSON objects with nested objects and arrays.
Data enrichment - As raw data streams into the platform, user-defined logic is executed to convert it, within milliseconds, into a cleansed, consistent, analytic data set enriched with computed values, aggregates and views.
Real-time data analysis - Powered by CrateDB, Postgres-compatible SQL syntax supports real-time time series, location, text search, and user-defined machine learning analyses.
Data visualization - Grafana, Klimt, or any other user-specified data visualization software can be hosted and scaled within the platform.
Operational triggers and alerting - User-defined rules trigger workflows and alerts that can improve operational efficiency in the moment.
Data Platform Administration highlights:
Platform scaling and orchestration - The Crate Machine Data Platform has a cloud-native microservices architecture, managed and secured via a Kubernetes environment.
Data aging and archiving - As data ages and loses relevance (especially in real-time systems), the platform automatically can archive data in inexpensive cold storage. It can be restored easily in order to perform new analyses over historical data.
Data backup and restore - Incremental database backups are automatically archived.
Monitoring and logging - All access to the platform is logged and can be monitored in real time.
Data security - Role-based access control is provided by the platform, and all access from within and without is encrypted and authenticated.
Supportive Funding Round
Crate.io announced $11 million in series A funding, led by Zetta Ventures with participation from Deutsche Invest Equity, Chalfen Ventures, Momenta Partners and Charlie Songhurst. Existing investors Draper Esprit, Vito Ventures and Solomon Hykes also participated in the round.

Crate.io has 30 paying customers, plus some 500 more using the open source CrateDB version around the world. Adoption is growing, which led to the just-completed funding, said Ellicott.

Crate.io will use the new money supply to accelerate development and adoption of both the commercial and open source offerings. That funding will expand the capabilities of CrateDB well beyond its roots, said Crate.io CEO Lutz.

"The introduction of the Crate Machine Data Platform and the innovations introduced with CrateDB 3.0 truly move us up the stack to enterprise actionability, making us a critical component of their data infrastructure and positioning us and them well for the future," he maintained.

The Crate Machine Data Platform and CrateDB 3.0 are available immediately.

Safe Growth Potential
Crate's expansion into a commercial offering enhances the success of the free open source database, according to Zededa's Shaposhnik. It strengthens the company's position.

"In the world of enterprise open source, there is nothing like having vendors focus on a particular set of customer needs, thus bringing those concerns back into the open source innovation space for consideration," he said.

Crate Machine Data Platform is exactly the right way for the company to focus on the two key issues that would allow actual business growth, he suggested. It will work for the company even if the evolution of the open source technology itself should decelerate.