This topic describes how to generate vector embeddings using external embedding models, including locally pre-trained models and online embedding services.
In addition to using the built-in Use the AI_EMBED function of OceanBase AI Database, you can also use external embedding models to generate vector embeddings at the application layer and then store the vector data in the database. This approach provides greater flexibility and a wider selection of models.
Procedure
You must have installed the pip command in advance.
Common text embedding methods
Use offline, local pre-trained embedding models
Using pre-trained models for text embedding locally is the most flexible approach, but it requires significant computational resources. Commonly used models include:
Use Sentence Transformers
Sentence Transformers are a model for natural language processing (NLP) designed to convert sentences or paragraphs into vector embeddings. They are based on deep learning techniques, particularly the Transformer architecture, and can effectively capture semantic information in text. Set the Hugging Face mirror address in advance: export HF_ENDPOINT=https://hf-mirror.com and then run the following code.
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("BAAI/bge-m3")
sentences = [
"That is a happy person",
"That is a happy dog",
"That is a very happy person",
"Today is a sunny day"
]
embeddings = model.encode(sentences)
print(embeddings)
# [[-0.01178016 0.00884024 -0.05844684 ... 0.00750248 -0.04790139
# 0.00330675]
# [-0.03470375 -0.00886354 -0.05242309 ... 0.00899352 -0.02396279
# 0.02985837]
# [-0.01356584 0.01900942 -0.05800966 ... 0.00523864 -0.05689549
# 0.00077098]
# [-0.02149693 0.02998871 -0.05638731 ... 0.01443702 -0.02131325
# -0.00112451]]
similarities = model.similarity(embeddings, embeddings)
print(similarities.shape)
# torch.Size([4, 4])
Use Hugging Face Transformers
Hugging Face Transformers is an open-source library that provides a large number of pre-trained deep learning models, especially those for natural language processing (NLP) tasks. Due to regional issues, accessing Hugging Face's domain name directly may time out. Set the Hugging Face mirror address in advance: export HF_ENDPOINT=https://hf-mirror.com and then run the following code.
from transformers import AutoTokenizer, AutoModel
import torch
# Load models and tokenizers
tokenizer = AutoTokenizer.from_pretrained("BAAI/bge-m3")
model = AutoModel.from_pretrained("BAAI/bge-m3")
# Prepare to Enter
texts = ["This is sample text"]
inputs = tokenizer(texts, padding=True, truncation=True, return_tensors="pt")
# Generate Embeddings
with torch.no_grad():
outputs = model(**inputs)
embeddings = outputs.last_hidden_state[:, 0] # The output of the [CLS] token
print(embeddings)
# tensor([[-1.4136, 0.7477, -0.9914, ..., 0.0937, -0.0362, -0.1650]])
print(embeddings.shape)
# torch.Size([1, 1024])
Ollama
Ollama is an open-source model. At runtime, it enables users to easily run, manage, and use various large language models locally. In addition to supporting open-source language models like Llama 3 and Mistral, it also supports embedding models like bge-m3.
Deploy Ollama
On macOS and Windows, you can directly download the installation package from the official website. For installation instructions, refer to the Ollama official website. After installation, Ollama can run as a backgound service.
To install Ollama on Linux:
curl -fsSL https://ollama.ai/install.sh | shPull the embedding model
Ollama supports using the bge-m3 model for text embedding:
ollama pull bge-m3Use Ollama for text embedding
You can use Ollama's embedding capabilities via HTTP APIs or Python SDKs:
HTTP API method
import requests def get_embedding(text: str) -> list: """Use the Ollama HTTP API to obtain text embeddings""" response = requests.post( 'http://localhost:11434/api/embeddings', json={ 'model': 'bge-m3', 'prompt': text } ) return response.json()['embedding'] # Example usage text = "This is an example text" embedding = get_embedding(text) print(embedding) # [-1.4269912242889404, 0.9092104434967041, ...]Python SDK method
First, install the Ollama Python SDK:
pip install ollamaThen use it as follows:
import ollama # Example usage texts = ["The first sentence", "The second sentence"] embeddings = ollama.embed(model="bge-m3", input=texts)['embeddings'] print(embeddings) # [[0.03486196, 0.0625187, ...], [...]]
Advantages and limitations of Ollama
Advantages:
- Fully local deployment, no network connection required
- Open-source and free, no API Key needed
- Supports multiple models, easy to switch and compare
- Relatively low resource consumption
Limitations:
- Fewer embedded model options
- Performance may not match commercial services
- Requires self-maintenance and updates
- Lacks enterprise-level support
When deciding whether to use Ollama, you need to weigh these factors. If your application has high privacy requirements or you want fully offline operation, Ollama is a good choice. However, if you need more stable service quality and better performance, you may still want to consider commercial services.
Use online, remote embedding services
Offline, local embedding models usually require a relatively high-spec machine and involve greater management overhead for model loading and unloading. Since many users need online embedding services, many AI inference service providers now offer corresponding text embedding services. Taking Qwen's text embedding service as an example, after first registering for an Alibaba Cloud Model Studio account and obtaining the API Key, you can then call its public interface to get text embedding results.
HTTP call
After obtaining the API Key, you can try text embedding using the following code. If the requests package is not installed in your Python environment, you need to install it first with pip install requests to send network requests.
import requests
from typing import List
class RemoteEmbedding():
def __init__(
self,
base_url: str,
api_key: str,
model: str,
dimensions: int = 1024,
**kwargs,
):
self._base_url = base_url
self._api_key = api_key
self._model = model
self._dimensions = dimensions
"""
OpenAI compatible embedding API. Tongyi, Baichuan, Doubao, etc.
"""
def embed_documents(
self,
texts: List[str],
) -> List[List[float]]:
"""Embed search docs.
Args:
texts: List of text to embed.
Returns:
List of embeddings.
"""
res = requests.post(
f"{self._base_url}",
headers={"Authorization": f"Bearer {self._api_key}"},
json={
"input": texts,
"model": self._model,
"encoding_format": "float",
"dimensions": self._dimensions,
},
)
data = res.json()
embeddings = []
try:
for d in data["data"]:
embeddings.append(d["embedding"][: self._dimensions])
return embeddings
except Exception as e:
print(data)
print("Error", e)
raise e
def embed_query(self, text: str, **kwargs) -> List[float]:
"""Embed query text.
Args:
text: Text to embed.
Returns:
Embedding.
"""
return self.embed_documents([text])[0]
embedding = RemoteEmbedding(
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1/embeddings",
api_key="your-api-key", # Enter your API Key
model="text-embedding-v3",
)
print("Embedding result:", embedding.embed_query("The weather is nice today"), "\n")
# Embedding result: [-0.03573227673768997, 0.0645645260810852, ...]
print("Embedding results:", embedding.embed_documents(["The weather is nice today", "What about tomorrow?"]), "\n")
# Embedding results: [[-0.03573227673768997, 0.0645645260810852, ...], [-0.05443647876381874, 0.07368793338537216, ...]]
Use Qwen SDK
Qwen provides a SDK called dashscope for quickly invoking model capabilities. After installing it with pip install dashscope, you can obtain text embeddings.
import dashscope
from dashscope import TextEmbedding
# Set API Key
dashscope.api_key = "your-api-key"
# Prepare the text to be entered
texts = ["This is the first sentence", "This is the second sentence"]
# Call the embedded service
response = TextEmbedding.call(
model="text-embedding-v3",
input=texts
)
# Get Embedding Result
if response.status_code == 200:
print(response.output['embeddings'])
# [{"embedding": [-0.03193652629852295, 0.08152323216199875, ...]}, {"embedding": [...]}]
Common image embedding methods
Use offline, local pre-trained embedding models
Use CLIP
CLIP (Contrastive Language-Image Pretraining) is a model proposed by OpenAI that aims to perform multimodal learning by combining images and text. CLIP can understand and process the relationships between images and text, enabling it to perform well in various tasks such as image classification, image search, and text generation.
from PIL import Image
from transformers import CLIPProcessor, CLIPModel
model = CLIPModel.from_pretrained("openai/clip-vit-base-patch32")
processor = CLIPProcessor.from_pretrained("openai/clip-vit-base-patch32")
# Prepare the input image.
image = Image.open("path_to_your_image.jpg")
texts = ["This is the first sentence", "This is the second sentence"]
# Call the embedded service
inputs = processor(text=texts, images=image, return_tensors="pt", padding=True)
outputs = model(**inputs)
# Get Embedding Result
if outputs.status_code == 200:
print(outputs.output['embeddings'])
# [{"embedding": [-0.03193652629852295, 0.08152323216199875, ...]}, {"embedding": [...]}]