This topic describes the concepts and types of dense vector indexes in OceanBase AI Database. Hereinafter, they are referred to as dense indexes.
Dense indexes are efficient index types provided by OceanBase AI Database for dense vector data (vectors where most elements are non-zero values). Dense vector indexes are primarily used to accelerate vector search and support nearest neighbor searches using distance functions (L2, inner product, cosine distance).
Index types
Dense indexes in OceanBase AI Database mainly include the HNSW series and the IVF series.
HNSW series
A memory-based graph-structured index that requires complete loading into memory. It supports DML and real-time search, offering high search performance and recall.
Index type |
Description |
|---|---|
| HNSW | The maximum dimension of an index column is 4,096. HNSW indexes are in-memory indexes that must be fully loaded into memory. |
| HNSW_SQ | The HNSW_SQ index provides similar construction speed, search performance, and recall to the HNSW index, but reduces the total memory usage to 1/2 to 1/3 of the original. |
| HNSW_BQ | The recall of the HNSW_BQ index is slightly lower than that of the HNSW index, but it significantly reduces memory usage. The BQ quantization compression algorithm (Rabbitq) can compress a vector to 1/32 of its original size. As the vector dimension increases, the memory optimization effect of the HNSW_BQ index becomes more pronounced. |
For more information, see the related documentation at the end of this topic.
IVF series
A disk-based index that does not occupy resident memory, making it suitable for large data volumes and cost-sensitive scenarios.
Index type |
Description |
|---|---|
| IVF | The IVF index implemented based on database tables does not occupy resident memory. |
| IVF_PQ | The IVF_PQ index implemented based on database tables does not occupy resident memory. It applies PQ quantization technology on top of IVF. The recall of this index is slightly lower than that of the IVF index, but its performance is higher. Additionally, the PQ quantization compression algorithm can typically compress a vector to 1/16 to 1/32 of its original size in most scenarios. |
Syntax description
HNSW series indexes support two creation methods: creation during table creation and post-creation; IVF series indexes support only post-creation. The search syntax and deletion syntax are consistent with those of regular vector indexes. For the creation syntax, parameter descriptions, search syntax, and examples of different index types, see:
- HNSW series indexes, which includes creation, search, and deletion syntax descriptions and examples for HNSW, HNSW_SQ, and HNSW_BQ.
- IVF series indexes, which includes creation, search, and deletion syntax descriptions and examples for IVF and IVF_PQ.
Related documentation
- For index selection recommendations, see Index type selection.
- Vector index memory management
- Monitor and maintain indexes