Vector Search in Apache Cassandra for RAG Retrieval
Semantic retrieval using one Cassandra table to understand how it works, where it shines, and what to watch for.
Vector search in Apache Cassandra ® lets teams store embeddings, index them with Storage-Attached Indexing, and run approximate nearest neighbor queries directly in CQL without managing a separate vector database. For RAG retrieval, that means Cassandra can act as both the operational database and the semantic search layer when the retrieval problem is primarily meaning-based.
Search used to require the person asking to already know the answer’s vocabulary. A technician precisely asks, “Recall 24V-113: tailgate latch”; a truck owner asks, “Is there a recall on the tailgate?” Embeddings close that gap by mapping meaning into geometry, so similar ideas land near each other in vector space regardless of the words used. That geometry is the retrieval layer underneath most AI applications. Using retrieval-augmented generation (RAG), a pattern for answering questions from supplied data instead of the model training set alone, the chatbot can be instructed to only answer from the documents vector search hands it. As a result, the retrieval quality sets the ceiling on the answer.
The standard approach utilizes a dedicated vector store alongside the operational database. While Cassandra is not the first operational database to keep embeddings next to the data they describe, version 5.0 made it a native part of the data model. When the use case fits, querying embeddings on the same infrastructure that houses the data can cut both complexity and cost: one database to operate instead of two, and no application code responsible for keeping them in sync.
This is Part 1 of 2. Part 1 demonstrates semantic retrieval against a realistic knowledge base containing vehicle recalls, tow ratings, and warranty terms using Cassandra. Then, we show where Approximate Nearest Neighbor (ANN) alone is not the complete answer, which sets up a different approach, hybrid RAG, in Part 2.
What is vector search on Apache Cassandra?Vector search on Apache Cassandra lets you store, index, and
query high-dimensional embeddings natively in the database using
the vector<float, N> column type and
Storage-Attached Indexing. Introduced in Cassandra 5.0, it
enables ANN search through an ORDER BY … ANN
OF clause in CQL, with no separate vector store.
At write time, title + body is embedded and stored in a vector
column on the same row as the text. At query time the question is
embedded with that same model, and Cassandra is asked for the
nearest stored vectors. Three pieces of CQL do the whole job:
declare the column, attach an SAI index so ANN is possible, and
query with ORDER BY … ANN OF plus an optional
similarity score in the select list.
CREATE TABLE docs ( id text PRIMARY KEY, title text, body text, category text, model text, model_year int, embedding vector<float, 384> ); CREATE INDEX ON docs (embedding) USING 'sai' WITH OPTIONS = {'similarity_function': 'COSINE'}; SELECT id, title, category, model, model_year, similarity_cosine(embedding, ?) AS score FROM docs ORDER BY embedding ANN OF ? LIMIT 5;
Each piece of that, in turn:
vector<float, N>is a native CQL column type holding an embedding of N dimensions, sitting on the row it describes rather than in another system. It’s 384 here becauseall-MiniLM-L6-v2(which is used for the demo application) produces 384-dimensional vectors; document and query vectors must always use the same model and dimension.- SAI (Storage-Attached Indexing) attaches a searchable index to the table as rows are written. For vector columns it plugs in JVector, an ANN engine that walks a graph of nearby embeddings for fast top-k retrieval.
-
similarity_functionacceptsCOSINE(the default),DOT_PRODUCT, orEUCLIDEAN. Changing it later means dropping and recreating the index. ORDER BYembeddingANN OF ?returns approximate nearest neighbors ordered by similarity. This is fast, but not a guarantee of the true global nearest neighbor.similarity_cosine(embedding, ?)is optional and exists to make the ranking legible. It returns a 0-to-1 value rather than raw cosine, so 0.5 means a document unrelated and a score of 0.75 is not “three-quarters relevant.”
Writes don’t need special handling because the embedding binds
as one more parameter on an ordinary
prepared INSERT alongside the document, and
SAI indexes it as the row is written, under the replication factor
and consistency level you already use. In addition, vectors impose
no special replication requirements. The demo application’s
keyspace is SimpleStrategy with RF 1 only because it’s a
single-node Docker cluster.
Throughout this series, you own a fictional 2024 Summit 1500 and you’re asking a support bot a question. The knowledge base is fictional but realistic: 21 documents covering 12 recalls, tow ratings, payload figures, maintenance intervals, and warranty terms for Summit 1500 and 2500 pickups.
The recall documents matter more than their count suggests. They share one sentence skeleton, mimicking how a real recall notice would read, making them a clean test of something specific: whether an embedding can resolve an identifier when the surrounding prose is nearly identical.
Everything in this series runs on Apache Cassandra 5.0.9 in
Docker, with embeddings generated locally from the all-MiniLM-L6-v2, an
embedding model, so there’s no API key to supply.
Companion app: cassandra-vector-demo.
Follow the instructions in the README to reproduce the output
below. Running the application isn’t required to understand the
concepts in this series.
Throughout this demo, you never directly query Cassandra. Rather, you’ve asked a question, expect an answer back, and never see the retrieved document’s similarity scores. Vector search is the middle step that chooses which rows the language model is allowed to read:
- The application takes the question as text.
- The same embedding model used at insert time turns that question into a vector.
- Cassandra returns a small list of nearest documents
(
ORDER BYembeddingANN OF … LIMIT k). - The application copies those titles and bodies into a prompt, next to the original question. That block is the context.
- The language model writes an answer using (in a well-behaved RAG setup) only that context. It doesn’t scan the knowledge base, only the handful of chunks retrieval picked.
We run three Summit owner questions against the table above and
read what Cassandra ranks. The demo doesn’t call an LLM after
retrieving the results; the point is whether they are trustworthy
before an LLM ever sees it. The first question is a paraphrase,
which is what embeddings exist to solve. The other two are the
limits you plan around: an exact identifier, and a question too
vague to have one answer. Both are properties of similarity search
rather than of Cassandra; the same model against Pinecone,
pgvector, or OpenSearch kNN should produce the same or similar
ordering and both are fixed with a query change rather than a
database change. The output below is generated
from src/demo.py against Cassandra 5.0.9
with all 21 documents seeded.
“Is there a recall on the tailgate?”
python src/demo.py "Is there a recall on the tailgate?"QUERY: Is there a recall on the tailgate? 1. [recalls] recall-24v-113 [1500 2024] Recall 24V-113 tailgate latch score=0.7767 2. [recalls] recall-23v-011 [1500 2023] Recall 23V-011 instrument cluster score=0.7032 3. [recalls] recall-23v-088 [1500 2023] Recall 23V-088 backup camera score=0.6935 4. [recalls] recall-23v-145 [2500 2023] Recall 23V-145 brake booster score=0.6934 5. [recalls] recall-22v-078 [2500 2022] Recall 22V-078 trailer brake module score=0.6808
Vector search ranks Recall 24V-113 (tailgate latch) first, and the margin is wide: 0.7767 against 0.7032 for the runner-up. Nobody wrote the word “tailgate” into the question expecting a latch recall, and the model bridged it anyway, against 11 other recall documents written in nearly identical prose. Everyday paraphrase is exactly what embeddings are for, and one CQL query against one table is the whole implementation.
Outcome 2: An exact recall ID lands nowhere near the top“What is recall 24V-330?”
Now you have the recall number.
python src/demo.py "What is recall 24V-330?" QUERY: What is recall 24V-330? 1. [recalls] recall-23v-011 [1500 2023] Recall 23V-011 instrument cluster score=0.7871 2. [recalls] recall-24v-207 [1500 2024] Recall 24V-207 fuel pump relay score=0.7701 3. [recalls] recall-23v-088 [1500 2023] Recall 23V-088 backup camera score=0.7623 4. [recalls] recall-24v-113 [1500 2024] Recall 24V-113 tailgate latch score=0.7560 5. [recalls] recall-24v-155 [2500 2024] Recall 24V-155 glow plug controller score=0.7514
The document for 24V-330 sits at rank 9, scoring 0.7332, below eight notices that never mention it. The recall notices share one sentence skeleton, so the ID is a handful of characters inside an otherwise near-identical embedding, and ANN ranks by overall closeness. Cosine distance has no notion of an exact token match, which is why the same model produces this ranking in any vector store, and why the fix is a lexical path beside ANN rather than a better embedding model. Part 2 of the series works to address this limitation.
Outcome 3: A question retrieval cannot answer as asked“How much can the Summit 1500 tow?”
python src/demo.py "How much can the Summit 1500 tow?" QUERY: How much can the Summit 1500 tow? 1. [towing] summit-1500-2024-tow [1500 2024] Tow ratings by configuration, 2024 model year score=0.8990 2. [towing] summit-1500-2023-tow [1500 2023] Tow ratings by configuration, 2023 model year score=0.8853 3. [towing] summit-2500-2024-tow [2500 2024] 2024 Summit 2500 heavy duty towing capacity score=0.8791 4. [specs] summit-1500-2024-payload [1500 2024] 2024 Summit 1500 payload capacity score=0.8276 5. [towing] hitch-classes Hitch receiver classes and weight distribution score=0.7550
The ranking looks reasonable until you read the bodies. The top four sit within 0.08 of each other and answer four different questions: 11,300 lb for the 2024 1500, 9,100 lb for the 2023 1500, 17,600 lb for the 2500, and 1,750 lb of payload, which isn’t a tow rating at all.
Retrieval did its job since all four are genuinely relevant to
what was asked. The question just never supplied a model year, so
nothing in the index can pick a winner. The fix is
a WHERE model_year = ? alongside ANN, which
is exactly why those columns are on the table, and will be utilized
in Part 2.
Vector search is enough when the question is about meaning
and an incorrect response is recoverable: everyday
paraphrase like “is there a recall on the tailgate,” similar
tickets, or similar products already sitting in Cassandra. It isn’t
enough when an identifier, a model year, or a customer record has
to be exact, and no similarity threshold will separate those two
cases for you. Those need a lexical path or
a WHERE clause next to ANN, which is the
work of Part 2 using additional SAI indexes on this same table, so
the fix stays inside Cassandra rather than adding the second system
you just avoided.
Neither of those statements is specific to Cassandra. They describe ANN retrieval wherever it runs, and teams building RAG on a dedicated vector store can reach the same two conclusions and apply the same two fixes. What Cassandra changes is the operational surface rather than the ranking: a column and an index on a table you already replicate, back up, and secure.
Native doesn’t mean equivalent. A dedicated search platform still wins on lexical ranking and relevance tuning, but it changes the starting assumption and depending on the use case, a second system doesn’t need to be a default to accept.
Apache Cassandra vector search version and operational notesVector search on Cassandra is continuing to mature, so treat patch releases as part of planning.
- Prefer 5.0.7 or later. Earlier
5.0 patches could return stale neighbors after overwrites and
tombstones, and
WHEREplus ANN queries were prone to timeouts. CASSANDRA-20086 reworked that path in 5.0.7 and 6.0. This project uses 5.0.9. Note that Instaclustr does not support 5.0.7 nor 5.0.8. - Cassandra 6.0 isn’t GA yet. Alpha1 arrived in March 2026, alpha2 in August. The work since 5.0 has been correctness rather than new capability, so nothing above changes.
- Cassandra 5.0.6 is Generally Available on the NetApp Instaclustr Managed Platform. This works for the demo, which is read-only against a freshly seeded table. Track the platform version list before putting vector writes into production.
Run the three questions against a live Cassandra 5 table on Docker. The demo application is from cassandra-vector-demo. Clone it, follow the README, and compare your rankings to the output in this series.
The same CQL runs on Instaclustr for Apache Cassandra 5.0. A new cluster takes a few clicks in the console; start a free trial and point the demo at the managed contact point instead of 127.0.0.1. You write the table; NetApp manages the cluster.
Further reading on native vectors: Apache Cassandra 5.0 vector search, similarity search with the vector type.
Part 2 of this series continues with the same Summit table and uses hybrid RAG to improve correctness.
The post Vector Search in Apache Cassandra for RAG Retrieval appeared first on Instaclustr.