The Embedding Model Starter Pack (and When to Actually Use Each One)

Photo by Iván Díaz on Unsplash

Embedding models are the bedrock of your RAG system. If your embeddings suck, it doesn’t matter how good your language model is — your chatbot will sound like it’s having a stroke.

But here’s the problem: there are too many damn embedding models, and most blog posts list them without telling you when or why to use each.

Wait, what even is an embedding model?

Embedding models take words, sentences, or documents and turn them into vectors — lists of numbers that represent their “meaning” in a math kind of way.

The idea? Similar meanings → closer vectors.

If you ask, “How do I boil an egg?”, you’d hope it ends up near something like “Steps to cook an egg” and not “File your taxes in Delaware.” Your embedding model is the one deciding those connections.

Common Embedding Models and When to Use Them

1. OpenAI’s text-embedding-3-small / text-embedding-3-large

Speed: Cloud only

Quality: Good

Use Case: SaaS apps, general-purpose RAG, search features in GPT-powered tools.

Why use it:

This is the gold standard for plug-and-play high-quality embeddings. Great semantic understanding, multilingual, battle-tested at scale.

Downside:

• Locked to OpenAI’s cloud.
• You’re paying per token.
• No customization. No peeking under the hood.

Pro tip:

Use 3-small for fast prototyping. Upgrade to 3-large when accuracy matters and you’re ready to pay for it.

2. all-MiniLM-L6-v2 (from SentenceTransformers)

Speed: Local or cloud, superfast

Quality: (surprisingly solid)

Use Case: Chatbots, internal search tools, MVPs, side projects

Why use it:

This little model punches way above its weight. For English queries and standard corpora, it’s good enough for 80% of use cases — and you can run it on a laptop.

Downside:

• Not great for niche topics.
• Doesn’t “get” long or very nuanced text.
• English-focused (multilingual variants exist but aren’t as polished).

When to use it:

If you’re launching something fast and cheap or want something you can self-host with minimal drama.

3. Cohere Embed v3 (Multilingual)

Speed: Cloud-only

Quality: Fair

Use Case: Apps with international users, translation-agnostic search

Why use it:

It’s trained across 100+ languages and doesn’t care if your query is in Japanese and your doc is in Portuguese. It’ll match them anyway.

Downside:

• Cloud locked
• Not as tuned to niche domains (e.g., medical/legal)
• The pricing model varies by plan.

Cool feature:

Supports query/document-specific embedding types (e.g., embed-qa, embed-classification, etc.).

4. E5 (intfloat/e5-base-v2)

Speed: Moderate (but self-hostable)

Quality: Good

Use Case: QA-heavy pipelines, technical docs, medium-scale enterprise search.

Why use it:

E5 is trained specifically for question-answer semantic similarity. That means it works well when your queries are in question form and your docs are explanatory.

Downside:

• Not great for zero-shot keyword-style search
• Still needs chunking to handle long docs.

Pro tip:

E5 likes this format: Query: What is diabetes? Vs. passage: Diabetes is a condition… — this improves performance significantly.

5. Instructor Models (hkunlp/instructor-xl)

Speed: (XL model is huge)

Quality: (for complex instructions)

Use Case: Research, nuanced search, reasoning-heavy RAG pipelines

Why use it:

These models are instruction-tuned, meaning they respond well to prompts like “Retrieve documents that explain the causes of climate change.” Great for when your queries are more than just keywords.

Downside:

• Resource-intensive
• Slower
• Requires thoughtful query formatting

Use it if:

You’ve got complex queries, smart users, or research-type use cases, and you want high relevance, not speed.

Usage Scenarios: The Quick & Dirty Map

Press enter or click to view image in full size

Most teams pick an embedding model once, jam it into their pipeline, and forget it exists. But if you’re serious about performance — accuracy, relevance, and trust — you’ve got to match the model to the mission.

Here’s how to think about it:

• Fast & good enough? → MiniLM
• Money to spend, want quality? → OpenAI 3-large
• Multilingual? → Cohere or LaBSE
• Building a vertical product? → Fine-tune, even just a little.
• Question-heavy search? → E5
• Complex intent search? → Instructor

Embeddings aren’t magic. But they’re foundational. If you’re getting garbage in your search results, start by checking what model translated your language into numbers. It’s probably there where the wires got crossed.