This topic describes how to store unstructured, semi-structured, and structured data in OceanBase AI Database. This not only fully leverages the foundational capabilities of OceanBase AI Database but also provides strong support for hybrid search.
Storage principle
OceanBase AI Database can store data in different modalities. It supports hybrid search by converting various types of data (such as text, images, and videos) into vectors, and performing searches by calculating the distances between these vectors. Hybrid search can be divided into two types: simple search, which is a similarity search based on a single vector; and complex search, which combines vector and scalar searches.
Since vector search itself is approximate, various methods must be combined in practical applications to improve accuracy. Only precise search results can bring greater value to your business.
Configure vector index memory
In OceanBase AI Database, vector search uses the ob_vector_memory_limit_percentage parameter to configure the memory for vector indexes:
The default value of this parameter is 0, which indicates adaptive mode. In adaptive mode, the system automatically adjusts the memory usage ratio for vector index data within an instance, eliminating the need for manual adjustment:
- When the actual instance memory is 8 GB or less, this value adapts to
40. - When the actual instance memory is more than 8 GB, this value adapts to
50.
Create a vector column
The following example shows a table that stores vector data, spatial data, and relational data simultaneously. The data type for the vector column is VECTOR, and its dimension must be specified during creation, with a maximum supported dimension of 16,000. The data type for the spatial column is GEOMETRY:
CREATE TABLE t (
-- Store relational data (structured data).
id INT PRIMARY KEY,
-- Store spatial data (semi-structured data).
g GEOMETRY,
-- Store vector data (unstructured data).
vec VECTOR(3)
);
Insert vector data using the INSERT statement
After you create a table that contains a column of the VECTOR data type, you can use the INSERT statement to insert vectors into the table. When you insert data, the vector must have the same dimension as specified when the table was created. Otherwise, an error will be returned. This design ensures data consistency and query efficiency. Vectors are represented as standard floating-point number arrays. Each dimension must have a valid floating-point number. Here is a simple example:
INSERT INTO t (id, g, vec) VALUES (
-- Insert structured data
1,
-- Insert semi-structured data
ST_GeomFromText('POINT(1 1)'),
-- Insert unstructured data
'[0.1, 0.2, 0.3]'
);