Note
This feature is an experimental feature in the current version.
This topic describes how to configure indexes and embedding functions for Agent hybrid search scenario, and explains how to combine the two.
Schema and vector indexes
When creating a Collection, you can customize the index configuration using the schema parameter. The default configuration varies by mode:
use_namespace=False: By default, enables HNSW indexing (384-dimensions, cosine distance) and the ik tokenizer.use_namespace=True(this scenario): You must explicitly configureIVFConfiguration. Currently, onlytype="ivf_flat"is supported.
Notice
use_namespace=True, only dense vector indexes and full-text indexes are supported; sparse vector indexes are not supported.Schema structure
Schema is used to define the index configuration for a Collection and contains three configuration fields:
vector_index: Dense vector indexfulltext_index: Full-text indexsparse_vector_index: Sparse vector index
from pyseekdb import Schema, VectorIndexConfig, IVFConfiguration, FulltextIndexConfig
collection = client.create_collection(
name="my_col",
schema=Schema(
# Vector Index
vector_index=VectorIndexConfig(
ivf=IVFConfiguration(
dimension=1024,
distance="cosine",
type="ivf_flat",
),
),
# Full-text index
fulltext_index=FulltextIndexConfig(analyzer="ik"),
),
use_namespace=True,
)
Chained API calls
Schema supports method chaining, allowing you to call methods one after another to gradually add various index configurations. For example, you can first create an empty object with Schema(), then continuously use .create_index(...) to add a vector index and a full-text index respectively. Each step returns a Schema object:
schema = (
Schema()
.create_index(IVFConfiguration(dimension=384, distance="cosine"))
.create_index(FulltextIndexConfig(analyzer="ik"))
)
create_index automatically mounts the corresponding index (IVFConfiguration / VectorIndexConfig / FulltextIndexConfig) based on the type of the passed object. The chained notation is equivalent to directly constructing Schema(...).
IVF vector index
IVFConfiguration(
dimension=384,
distance="cosine",
type="ivf_flat",
lib="ob",
centroids_fresh_mode="spfresh",
properties={"nprobes": 32},
)
Parameters |
Default Value |
Note |
|---|---|---|
| dimension | 384 | Vector dimension, which must match the output of the embedding model. Value range: [1, 4096]. |
| distance | "cosine" | Distance metric. Currently, only "l2", "cosine", and "inner_product" are supported. |
| type | "ivf_flat" | Index subtype. Currently, only "ivf_flat" is supported. |
| lib | "ob" | Underlying index implementation database. Currently, only "ob" is supported. |
| centroids_fresh_mode | None | SPFresh mode. Optional. Currently, only "spfresh" is supported. |
| properties | None | The index property dictionary, supporting parameters such as nprobes. |
Full-text index
The full-text index is configured using FulltextIndexConfig. It is used for the full-text search side of hybrid_search and for filtering by the $contains condition in where_document.
Basic configuration
FulltextIndexConfig(
analyzer="ik",
properties=None,
)
Tokenizer selection and properties
Supported tokenizers include: ik, space, ngram, ngram2, and beng. For specific descriptions and supported properties (properties), see: Full-text index - Field description.
Embedding function
An embedding function converts text into vectors. The interface for this feature is consistent with that of the EF in seekdb (default model, built-in implementation, custom protocol, and automatic vectorization rules are all the same). The only difference in syntax is that in this scenario (use_namespace=True), the EF must be configured in VectorIndexConfig(embedding_function=...), not as a top-level parameter for create_collection (the top-level parameter is ignored when passed to schema).
General instructions, platform authentication, and customization details are described in seekdb: Overview of embedding functions.
How it works
After configuring an embedding function, vectorization is automatically performed during write and query operations:
- Write: When
add,update, orupsertcalls passdocumentsbut noembeddings, the embedding function generates vectors. - Query: When
queryorhybrid_searchcalls passquery_textsbut noquery_embeddings, the embedding function generates query vectors.
Note
The IVFConfiguration.dimension in the index configuration must match the output dimensionality of the embedding function.
Configuration example
from pyseekdb import Schema, VectorIndexConfig, IVFConfiguration
from pyseekdb.utils.embedding_functions import OpenAIEmbeddingFunction
ef = OpenAIEmbeddingFunction(
api_key="sk-xxx", # or environment variable OPENAI_API_KEY
model_name="text-embedding-3-small",
)
collection = client.create_collection(
name="my_collection",
schema=Schema(
vector_index=VectorIndexConfig(
ivf=IVFConfiguration(dimension=1536, distance="cosine"),
embedding_function=ef, # Configure the embedding function; it must be written inside VectorIndexConfig.
),
),
use_namespace=True,
)
If embedding_function is not provided, the default DefaultEmbeddingFunction (all-MiniLM-L6-v2, dimension 384) is used (the model may be downloaded for the first time during local execution). For more information, see Default embedded function.
When vectors are passed manually, you can explicitly disable automatic vectorization:
collection = client.create_collection(
name="manual_vectors",
schema=Schema(
vector_index=VectorIndexConfig(
ivf=IVFConfiguration(dimension=384, distance="cosine"),
# Explicitly Close
embedding_function=None,
),
),
use_namespace=True,
)
Built-in and platform implementations
The dependencies and parameters for built-in implementations such as OpenAI, Qwen, Ollama, Cohere, and Bedrock are the same as those for seekdb. Please refer to Overview of embedding functions - Supported providers for details by platform. Configuration still requires placing it in this scenario's VectorIndexConfig.
Custom embedding function
For complete constraints and examples, see:
A configuration example for this scenario is as follows (search uses IVF, with dimensions aligned with ef_dimension):
from pyseekdb import (
EmbeddingFunction,
IVFConfiguration,
Schema,
VectorIndexConfig,
register_embedding_function,
)
@register_embedding_function
class MyEmbedding(EmbeddingFunction):
@staticmethod
def name() -> str:
return "my_embedding"
def __call__(self, documents):
# Return list[list[float]]
return [[0.1, 0.2, 0.3] for _ in documents]
def get_config(self) -> dict:
return {"model": "custom-v1"}
@staticmethod
def build_from_config(config: dict):
return MyEmbedding()
@staticmethod
def support_persistence(embedding_function) -> bool:
return False
# Usage
collection = client.create_collection(
"my_col",
schema=Schema(
vector_index=VectorIndexConfig(
ivf=IVFConfiguration(dimension=3),
embedding_function=MyEmbedding(),
),
),
use_namespace=True,
)
When customizing, you must inherit from EmbeddingFunction, implement at least the __call__ method to convert text into a vector, and it is recommended to declare dimension. If you want to ensure that the same set of instances is automatically restored when using get_collection next time after creating a Collection, implement name, get_config, and build_from_config, and register it with @register_embedding_function: The SDK stores the name and the return value of get_config() in the Collection metadata, and then rebuilds the instance with build_from_config.