Fixing FAISS AssertionError: Embedding Dimension 1536 vs 768

The Anatomy of a Dimension Mismatch

It usually happens right after a "quick" configuration change. You swap a local HuggingFace model for an OpenAI API call, restart your service, and your RAG (Retrieval-Augmented Generation) pipeline immediately crashes. This isn't a bug in your logic; it's a fundamental math failure.

AssertionError: Embedding dimension 1536 does not match index dimension 768.

FAISS is a high-performance C++ library that treats your data as points in a multi-dimensional grid. It demands strict consistency. If you built your index using all-mpnet-base-v2 (768 dimensions) but try to query it with OpenAI's text-embedding-3-small (1536 dimensions), the comparison is mathematically impossible. You cannot compare a point in 768-D space to one in 1536-D space.

Pinpointing the Conflict

Before deleting your data, identify the exact dimensions of your two components. This error typically surfaces because the embeddings object passed to FAISS.load_local() differs from the one used during FAISS.save_local().

Step 1: Check the Saved Index

You don't need to run a query to find the index's requirements. Inspect the .faiss file directly using the underlying library:

import faiss

# Load the raw index file
index = faiss.read_index("index_path/index.faiss")
print(f"Index dimension: {index.d}")
# Output: Index dimension: 768

Step 2: Verify Your Active Model

Next, confirm what your current code is actually generating. A single line of code can verify the output size of your embedding provider:

from langchain_openai import OpenAIEmbeddings

# Initialize your current model
embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
test_vector = embeddings.embed_query("test")
print(f"Model dimension: {len(test_vector)}")
# Output: Model dimension: 1536

If the numbers 768 and 1536 (or 384 and 1024) don't align, the AssertionError will persist.

How to Fix It

You have two choices: downgrade your model to match the old index or rebuild the index to leverage the new model. Most developers choose to rebuild if they are upgrading to a more capable LLM stack.

Option A: Rebuilding the Index (Recommended for Upgrades)

There is no way to "resize" an existing vector. If you move from a 768-dimension model to a 1536-dimension model, you must re-embed your source documents. It is a fresh start for your vector store.

from langchain_community.vectorstores import FAISS
from langchain_openai import OpenAIEmbeddings
from langchain_community.document_loaders import TextLoader
from langchain_text_splitters import CharacterTextSplitter

# 1. Use the new 1536-dimension model
embeddings = OpenAIEmbeddings(model="text-embedding-3-small") 

# 2. Process source documents
loader = TextLoader("knowledge_base.txt")
documents = loader.load()
text_splitter = CharacterTextSplitter(chunk_size=1000, chunk_overlap=50)
docs = text_splitter.split_documents(documents)

# 3. Create and save the new index
vectorstore = FAISS.from_documents(docs, embeddings)
vectorstore.save_local("faiss_index_v2")

Option B: Reverting the Model

If re-indexing millions of documents is too expensive or time-consuming, you must use the original model. For instance, if the index was built with all-MiniLM-L6-v2, ensure your loader uses exactly that:

from langchain_community.embeddings import HuggingFaceEmbeddings

# This model generates 384 dimensions
embeddings = HuggingFaceEmbeddings(model_name="sentence-transformers/all-MiniLM-L6-v2")

# Load the matching 384-dimension index
vectorstore = FAISS.load_local("legacy_index", embeddings, allow_dangerous_deserialization=True)

Future-Proofing Your RAG Pipeline

Prevent these crashes from reaching production by adding a validation check during the startup sequence. This script compares the model and index dimensions before the application starts accepting queries:

def safe_load_faiss(folder_path, embeddings_model):
    vectorstore = FAISS.load_local(folder_path, embeddings_model, allow_dangerous_deserialization=True)
    
    # Test a sample embedding against the index
    model_dim = len(embeddings_model.embed_query("validation"))
    index_dim = vectorstore.index.d
    
    if model_dim != index_dim:
        raise ValueError(f"Critical Mismatch: Model is {model_dim}D, but Index is {index_dim}D.")
    
    return vectorstore

Key Takeaways

• Explicit Naming: Name your index folders after the model and dimension (e.g., vector_store_openai_1536) to avoid confusion.
• Store Metadata: Keep a config.json file next to your FAISS index that records the model name, version, and dimension count.
• Watch for Defaults: OpenAI's text-embedding-3-small defaults to 1536, but it can be truncated. Always specify the dimension if you aren't using the default.