From Vectors to Answers: How Cosine Similarity Works in Databases

In the world of modern data storage and retrieval, vector databases have emerged as a powerful tool for handling high-dimensional data. Unlike traditional databases that store structured data in rows and columns, vector databases store data as vectors—mathematical representations of objects in a multi-dimensional space. This makes them particularly useful for applications like natural language processing (NLP), image recognition, and recommendation systems.

In this blog, we’ll explore how vector databases work, how data is stored, and how cosine similarity is used to retrieve relevant information. We’ll also walk through a practical example to solidify your understanding.

What is a Vector Database?

A vector database is designed to store and query vector embeddings. These embeddings are numerical representations of data, such as words, sentences, or images, in a high-dimensional space. For example, in NLP, words or sentences are converted into vectors using models like Word2Vec, GloVe, or BERT.

How Data is Stored in a Vector Database

Let’s say we have a sentence:

"The quick brown fox jumps over the lazy dog."

Using a pre-trained language model, we can convert this sentence into a vector. Suppose the vector representation of this sentence is:

v=[0.2,−0.1,0.4,…,0.7]

v=[0.2,−0.1,0.4,…,0.7]

Here, v is a high-dimensional vector (e.g., 300 dimensions) that captures the semantic meaning of the sentence.

In a vector database, this vector is stored alongside its metadata (e.g., the original sentence). When we query the database, we compare the query vector with the stored vectors to find the most similar ones.

Cosine Similarity: The Key to Retrieval

We need a way to measure the similarity between vectors to retrieve relevant data from a vector database. One of the most common methods is cosine similarity.

What is Cosine Similarity?

Cosine similarity measures the cosine of the angle between two vectors in a multi-dimensional space. It ranges from -1 to 1, where:

• 1 indicates that the vectors are identical,

• 0 indicates that the vectors are orthogonal (no similarity),

• -1 indicates that the vectors are opposed.

Mathematically, cosine similarity between two vectors A and B is defined as:

Where:

• A⋅B is the dot product of the vectors,

• ∥A∥ and ∥B∥ are the magnitudes (Euclidean norms) of the vectors.

Example: Using Cosine Similarity to Retrieve Data

Scenario

Imagine we have a vector database that stores sentences as vectors. We’ll use the following two sentences:

1. Stored Sentence: "I love programming in Python."

2. Query Sentence: "I enjoy coding in Python."

Our goal is to convert these sentences into vectors, store one in the database, and use cosine similarity to retrieve the stored sentence when queried with the other.

Step 1: Convert Sentences into Vectors

To keep things simple, let’s assume we’re using a basic embedding model that converts words into 3-dimensional vectors. Here’s how the words might be represented:

Word Vector Representation

I - [0.1, 0.2, 0.3]

love - [0.4, 0.5, 0.6]

programming - [0.7, 0.8, 0.9]

in - [0.2, 0.3, 0.4]

Python - [0.5, 0.6, 0.7]

enjoy - [0.3, 0.4, 0.5]

coding - [0.6, 0.7, 0.8]

Now, let’s convert the sentences into vectors by averaging the word vectors (a simple way to create sentence embeddings):

1. Stored Sentence: "I love programming in Python."

   

   1. Query Sentence: "I enjoy coding in Python."

      

      Now, we have:

      • Stored Vector: V1=[0.38,0.48,0.58]v1=[0.38,0.48,0.58]

      • Query Vector: V2=[0.34,0.44,0.54]v2=[0.34,0.44,0.54]

      ---

      Step 2: Compute Cosine Similarity

      The formula for cosine similarity is:

      

      Step 2.1: Compute the Dot Product (V1⋅V2)

      

      Step 2.2: Compute the Magnitudes (∥V1∥ and ∥V2∥)

      

      Step 2.3: Compute Cosine Similarity

      

      Step 3: Interpret the Result

      The cosine similarity between V1 and V2 is approximately 0.999, which is very close to 1. This indicates that the two sentences are almost identical in meaning. As a result, the vector database would retrieve the stored sentence "I love programming in Python." as a match for the query "I enjoy coding in Python."

      ---

      Why Does This Work?

      • The vectors V1 and V2 are very close in orientation because the sentences are semantically similar.

      • Cosine similarity ignores the magnitude of the vectors and focuses only on their direction, making it ideal for comparing high-dimensional data like text embeddings.

      ---

      Why Use Cosine Similarity?

      Cosine similarity is particularly useful in vector databases because:

      1. It focuses on the orientation of vectors rather than their magnitude, making it ideal for comparing high-dimensional data.

      2. It is computationally efficient and works well with sparse vectors (e.g., in NLP).

      3. It captures semantic similarity, which is crucial for applications like search engines and recommendation systems.

      ---

      Conclusion

      Vector databases, combined with cosine similarity, provide a robust framework for storing and retrieving high-dimensional data. By converting data into vectors and using mathematical techniques like cosine similarity, we can efficiently find relevant information even in vast datasets.

      Whether you’re building a search engine, a recommendation system, or an NLP application, understanding vector databases and cosine similarity is essential. So, the next time you encounter a vector database, remember: it’s all about the angles!