Introduction #
Retrieval-Augmented Generation (RAG) based on vector databases is widely used in production. A vector database primarily stores vectors, enabling precise similarity-based queries through vector operations, a foundational capability in various applications.
Among many vector database providers such as Weaviate, Qdrant, Milvus, and Chroma, we will focus exclusively on Qdrant, referencing the official Qdrant documentation (2025-03).
Basic Data Types #
Qdrant defines specific data abstractions to handle vector data effectively:
Point #
The fundamental unit of storage in a vector database. Typically, a point contains a vector and an optional payload with additional metadata:
Point = Vector + Payload
{
"id": 129,
"vector": [0.1, 0.2, 0.3, 0.4],
"payload": {"color": "red"}
}
Qdrant supports several vector types:
Type | Description |
---|---|
Dense vectors | Standard vectors generated by most embedding models. |
Sparse vectors | Variable-length vectors with few non-zero elements; useful for token matching and collaborative filtering. |
Multi vectors | Matrices composed of multiple dense vectors; each point may have different vector counts but identical dimensions. |
Named vectors | A mix of the above types, allowing various vector types within a single point. |
Payload #
Additional metadata stored with vectors using JSON syntax:
{
"name": "jacket",
"colors": ["red", "blue"],
"count": 10,
"price": 11.99,
"locations": [{"lon": 52.5200, "lat": 13.4050}],
"reviews": [
{"user": "alice", "score": 4},
{"user": "bob", "score": 5}
]
}
Payloads enhance semantic similarity searches by enabling additional logical filtering conditions.
Collection #
A group of points. Collections allow you to define:
- Similarity algorithms between points
- Vector dimensions
- Optimizer settings
- HNSW algorithm parameters
- Write-Ahead Log (WAL) configurations
- Quantization settings
Common Operations #
The essential operations for using vector databases:
Search #
Primarily refers to similarity search, leveraging the concept that objects with higher similarity are closer in vector space. Qdrant supports various similarity algorithms:
- Dot product
- Cosine similarity
- Euclidean distance
- Manhattan distance
Qdrant provides a comprehensive Search API, enabling:
- Basic searches with a vector or specifying a named vector.
- Control over search algorithms, including exact matching (slower but precise).
- Result filtering based on payload tags or similarity thresholds.
- Limiting search results (
limit
). - Batch searches for multiple vectors.
- Grouping search results (
group_size
). - Query planning for optimized performance based on heuristic evaluations.
group_size
and limit
, the limit
parameter indicates the number of groups.
Sparse vectors and dense vectors differ notably in search behavior:
Aspect | Sparse Vectors | Dense Vectors |
---|---|---|
Similarity method | Defaults to dot product, no need to specify | You can specify supported algorithms |
Search approach | Only exact search | HNSW for approximate searching |
Search results | Only vectors sharing common non-zero elements | Top limit vectors as specified |