DataStax makes it easier to build generative AI RAG apps with new data API

DataStax is looking to make it easier for developers to build generative AI retrieval augmented generation (RAG) applications with a new data API out today.

DataStax is one of the leading commercial vendors behind the open source Apache Cassandra database, which is the foundation of its AstraDB cloud database-as-a-service.  Like many other database vendors, DataStax has added vector database capabilities to its platform in 2023. At a recent event, DataStax’s CEO claimed that Cassandra was ,”..the best f*cking database for gen AI.”

Vector database capability is critical to enabling RAG applications which combine large language models (LLMs) with data platforms to generate highly accurate and customized results.  

While DataStax has had vector capabilities in AstraDB since July 2023, that capability still required users to work with the Cassandra Query Language (CQL) as the primary path to query the data. The new data API out today changes that, providing developers with the ability to use the  Python and JavaScript programming languages to access the database, which the company claims helps to narrow the gap between DataStax and purpose built vector database like Pinecone which just updated its namesake platform with serverless database functionality.

“There has been a kind of tug of war between the native vector databases that don’t support any other query type other than vectors and the hybrid databases that have very robust query models,” Ed Anuff, chief product officer at DataStax told TechForgePulse. “What we looked to do was to close that gap and that’s what the date API is all about.”

How the DataStax data API changes the way developer build RAG applications

The new data API does not provide any new vector capabilities to the AstraDB database. Instead what it does is make it easier for developers to build applications.

According to Anuff, the new API aims to reduce the impedance mismatch between what developers are doing and what the database provides. Anuff noted that since July of 2023 when the vector capabilities first landed in AstraDB approximately half of all new users that signed up for the cloud database are using it to build gen AI applications. 

The challenge is that those developers weren’t able to easily use the programming languages they were already using to build gen AI applications, which is largely Python and JavaScript, to access AstraDB.

Before the new data API, developers building AI applications with AstraDB would have had to use the standard Cassandra Query Language (CQL), which involves more data modeling knowledge than developers wanted to deal with for simple rack applications. The queries also would not have been as optimized for vector data.

Anuff explained that he new data API makes it easier by automatically handling vectorization, presenting a simpler interface in languages like Python and JavaScript, and optimizing performance by storing and indexing the vector data more efficiently at the database level rather than just adding vectors as another datatype. This reduces the learning curve and improves performance compared to just building on top of the existing Cassandra APIs and data model.

It’s all about APIs

With some classes of database APIs, all that occurs is a form of translation from a native programming language, like Python or JavaScript, into whatever the query language is for the database. That’s functionally very similar to a decades-old approach to how developers have worked with databases, via an Object Relational Mapper (ORM).

The DataStax data API is a bit different since Cassandra is architected differently than other databases.  Cassandra at the architecture level is organized around a set of high performance primitives that are combined together to support different types of query patterns. Anuff said that the Cassandra data architecture makes it possible to connect at a deeper layer in the database, which improves overall query performance.

“The data API exposes to the developer a very simple JSON based data format, where anything you can express within JSON, the developer can send and retrieve from the database,” Anuff said. “But we store that in a very efficient way within Cassandra where we do that directly at the storage tier and ensure that the performance that a developer gets is maintained.”

Accelerating vectors with JVector engine

Another key part of DataStax’s vector database advancement is the JVector search engine which is part of AstraDB.  JVector is an open source embedded vector search engine that was developed by DataStax.

Anuff explained that JVector uses an algorithm called DiskANN which is a disk-based storage optimized version of the ANN (approximate nearest neighbor search)  algorithm that is widely used across nearly all vector databases. He noted that DiskANN provides significantly better retrieval capabilities compared to other algorithms that don’t perform as well at large storage and distribution scales. 

According to DataStax, the JVector engine is what allows AstraDB to achieve better relevancy and recall than other vector databases. Much of DataStax’s vector work, including JVector and the data API are being open sourced to be used by the Cassandra open source community as well as DataStax’s AstraDB customers.

“We’re very strongly committed to making stuff available to open source ecosystems,” Anuff said. “We also just want to make sure that if you’re just the developer trying to figure out what cloud service you should use, that you’ve got the easiest path for that.”