Mastering Retrieval-Augmented Generation (RAG) and Prompt Design 

Below you will find some suggested materials on these topics, which we believe may help you complete the final activity in the lesson. Please note that using these resources is entirely optional, and you are welcome to explore any other sources or courses at your own discretion.

Introduction (2 Minutes)

Opening Statement:

• Welcome to Lesson 2 of Module 4. In this session, we will focus on how to connect FAISS, a powerful vector database, with Large Language Model (LLM) APIs like OpenAI or HuggingFace.

• By the end of this lesson, you’ll understand how FAISS integrates with LLMs to perform efficient context retrieval and enhance response generation.

Agenda:

1. Introduction to FAISS and its role in RAG.

2. Steps to connect FAISS with an LLM API.

3. Practical example: Querying the vector database for relevant context and generating responses.

Part 1: FAISS and Its Role in RAG (3 Minutes)

What is FAISS?

• FAISS (Facebook AI Similarity Search) is an open-source library designed for efficient similarity search and clustering of dense vectors.

• It is commonly used in Retrieval-Augmented Generation to find relevant data points (e.g., documents, profiles) based on a query.

Why Use FAISS?

• Efficiency: Handles millions of vectors with low latency.

• Scalability: Optimized for large-scale datasets.

• Flexibility: Supports various indexing methods and similarity metrics.

FAISS in RAG:

1. Stores embeddings of knowledge or data.

2. Retrieves the most relevant embeddings based on a user query.

3. Provides these embeddings as context to LLMs for response generation.

Part 2: Steps to Connect FAISS with an LLM API (5 Minutes)

Overview:

• The process involves embedding generation, building a FAISS index, querying for relevant embeddings, and integrating these with an LLM for generating responses.

1. Embedding Generation

• Convert data (e.g., text, employee profiles) into embeddings using an LLM API.

• Example APIs: OpenAI’s GPT-4 or HuggingFace’s Sentence Transformers.

Code Example:

python

Copy code

import openai

openai.api_key = "your_openai_api_key"

# Function to generate embeddings

def generate_embedding(text):

    response = openai.Embedding.create(

        input=text,

        model="text-embedding-ada-002"

    )

    return response['data'][0]['embedding']

2. Build a FAISS Index

• Store the embeddings in a FAISS index for similarity search.

Code Example:

python

Copy code

import faiss

import numpy as np

# Sample embeddings (replace with actual embeddings)

embeddings = np.array([[0.1, 0.2, 0.3], [0.3, 0.4, 0.5]])

# Define embedding dimension

dimension = embeddings.shape[1]

# Create a FAISS index

index = faiss.IndexFlatL2(dimension)

# Add embeddings to the index

index.add(embeddings)

3. Query the FAISS Index

• Generate an embedding for the query using the LLM API.

• Retrieve the most similar embeddings using FAISS.

Code Example:

python

Copy code

# Example query

query = "Who can help me with API integration?"

# Generate query embedding

query_embedding = np.array([generate_embedding(query)])

# Perform similarity search

k = 3  # Number of results

distances, indices = index.search(query_embedding, k)

print("Distances:", distances)

print("Indices:", indices)

4. Integrate with LLM for Response Generation

• Use the retrieved embeddings as context to generate a response.

Code Example:

python

Copy code

# Combine retrieved context

retrieved_context = " ".join(["Context from document {}".format(idx) for idx in indices[0]])

# Generate LLM response

prompt = f"Context: {retrieved_context}\n\nQuestion: {query}\n\nAnswer:"

response = openai.Completion.create(

    engine="text-davinci-003",

    prompt=prompt,

    max_tokens=100

)

print("Generated Response:", response['choices'][0]['text'].strip())

Part 3: Practical Example: Jarvid Use Case (8 Minutes)

Scenario:

• Query: “Who can help me with Python development?”

• Objective: Use Jarvid’s FAISS index of employee embeddings to find the most relevant profiles and generate a Slack response.

Step-by-Step Walkthrough:

1. Embedding Generation:

   • Generate embeddings for employee profiles using GPT-4.

2. Build FAISS Index:

   • Add these embeddings to the FAISS vector database.

3. Query Embedding:

   • Generate an embedding for the user query.

4. Retrieve Context:

   • Use FAISS to find the top 3 employee profiles.

5. Generate Response:

   • GPT-4 generates a Slack-compatible response summarizing the relevant profiles.

Code Example:

python

Copy code

# Employee profiles (sample data)

profiles = ["Alice: Python expert in AI Development",

            "Bob: Experienced in API integrations and Python",

            "Carol: Data Scientist with Python expertise"]

# Generate embeddings for profiles

profile_embeddings = np.array([generate_embedding(profile) for profile in profiles])

# Add to FAISS index

index.add(profile_embeddings)

# Query

query = "Who can help me with Python development?"

query_embedding = np.array([generate_embedding(query)])

distances, indices = index.search(query_embedding, 3)

# Generate Slack-compatible response

retrieved_profiles = [profiles[idx] for idx in indices[0]]

response_prompt = f"Profiles matching the query:\n" + "\n".join(retrieved_profiles)

response = openai.Completion.create(

    engine="text-davinci-003",

    prompt=response_prompt,

    max_tokens=100

)

print("Response for Slack:", response['choices'][0]['text'].strip())

Conclusion and Key Takeaways (5 Minutes)

Recap:

• • • FAISS acts as the retrieval engine for embeddings, ensuring efficient and scalable similarity searches.

    • Integrating FAISS with LLM APIs like OpenAI allows for dynamic and contextually accurate responses.

    • By combining embeddings and LLMs, systems like Jarvid can efficiently connect users to the right resources.

Next Steps:

• • • Experiment with different similarity metrics (e.g., cosine similarity).

    • Optimize the FAISS index for larger datasets using approximate nearest neighbor methods.

Objective

In this hands-on activity, you will:

1. 1. Load vectorized employee profiles into a FAISS index.

   2. Perform similarity searches using a sample query.

   3. Use the retrieved data to generate responses with an LLM AP

Step-by-Step Instructions

Step 1: Load Vectorized Employee Profiles

Dataset

Ensure you have a dataset with employee profiles and their embeddings. For this example, we’ll simulate vectorized profiles.

Sample Dataset:

python

Copy code

# Employee profiles with embeddings (sample data)

profiles = [

    {"id": 1, "name": "Alice", "Role": "Python Developer", "Department": "AI Development"},

    {"id": 2, "name": "Bob", "Role": "Data Scientist", "Department": "Data Analytics"},

    {"id": 3, "name": "Carol", "Role": "Backend Engineer", "Department": "API Integrations"},

]

# Simulated embeddings (replace with actual embeddings)

embeddings = [

    [0.1, 0.2, 0.3],  # Alice

    [0.3, 0.4, 0.5],  # Bob

    [0.5, 0.6, 0.7],  # Carol

]

Initialize FAISS

Create a FAISS index and add the embeddings.

Code Example:

python

Copy code

import faiss

import numpy as np

# Convert embeddings to a NumPy array

embeddings_array = np.array(embeddings, dtype=np.float32)

# Define embedding dimension

dimension = embeddings_array.shape[1]

# Create a FAISS index

index = faiss.IndexFlatL2(dimension)

# Add embeddings to the index

index.add(embeddings_array)

# Verify index content

print(f"Number of vectors in FAISS index: {index.ntotal}")

Step 2: Perform Similarity Searches

Simulate a User Query

Generate an embedding for the query using an LLM API.

Code Example:

python

Copy code

from openai import OpenAI

import numpy as np

# Function to generate query embedding

def generate_query_embedding(query):

    client = OpenAI(api_key="your_openai_api_key")  

    response = client.embeddings.create(

        input=query,

        model="text-embedding-ada-002"

    )

    return np.array(response.data[0].embedding, dtype=np.float32)

# Example query

query = "Who can help me with Python development?"

query_embedding = generate_query_embedding(query)

# Reshape for FAISS

query_embedding = query_embedding.reshape(1, -1)

Search the FAISS Index

Retrieve the most similar profiles.

Code Example:

python

Copy code

# Number of results to retrieve

k = 3

# Perform similarity search

distances, indices = index.search(query_embedding, k)

# Display results

print("Distances:", distances)

print("Indices:", indices)

# Map indices to profiles

retrieved_profiles = [profiles[idx] for idx in indices[0]]

print("Retrieved Profiles:", retrieved_profiles)

Step 3: Generate Responses with an LLM API

Integrate Retrieved Data

Combine retrieved profiles into a response format.

Code Example:

python

Copy code

# Build context from retrieved profiles

context = "\n".join([

    f"{profile['name']}, a {profile['Role']} in {profile['Department']}"

    for profile in retrieved_profiles

])

# Construct prompt

prompt = f"Context: {context}\n\nQuestion: {query}\n\nAnswer:"

Generate the Response

Use GPT-4 to generate a response based on the retrieved context.

Code Example:

python

Copy code

from openai import OpenAI

# Initialize the client

client = OpenAI(api_key="your_openai_api_key")  

response = client.chat.completions.create(

  model="gpt-3.5-turbo",

  messages=[

    {"role": "user", "content": final_prompt}

  ],

  max_tokens=100

)

# Display the response

print("Generated Response:")

print(response.choices[0].message.content.strip())

Step 4: Test and Verify

• Sample Query: "Who can help me with Python development?"

• Expected Output:

  • Retrieved Profiles: Alice (Python Developer), Bob (Data Scientist), etc.

  • Generated Response: "Alice and Bob are experts in Python development. You can reach out to them for assistance."

Bonus: Save and Reload the FAISS Index

To avoid rebuilding the index in the future, save it to a file.

Code Example:

python

Copy code

# Save FAISS index

faiss.write_index(index, "faiss_index.bin")

# Reload the index

index = faiss.read_index("faiss_index.bin")

print(f"Reloaded FAISS index with {index.ntotal} vectors.")

Expected Outcomes

1. Functional FAISS Index:

   • Stores employee embeddings and enables efficient similarity searches.

2. Similarity Search Results:

   • Retrieves the most relevant employee profiles based on a user query.

3. Generated LLM Response:

   • Provides a contextually relevant answer, integrating the retrieved profiles.

Step-by-Step Script

1. Setup

Install the necessary libraries:

bash

Copy code

pip install faiss-cpu openai pandas numpy

2. Script Code

python

Copy code

import faiss

import numpy as np

from openai import OpenAI

# Initialize OpenAI client

client = OpenAI(api_key="your_openai_api_key")

# Employee profiles and embeddings (replace with real data)

profiles = [

    {"id": 1, "name": "Alice", "role": "Python Developer", "department": "AI Development"},

    {"id": 2, "name": "Bob", "role": "Data Scientist", "department": "Data Analytics"},

    {"id": 3, "name": "Carol", "role": "Backend Engineer", "department": "API Integrations"},

]

# Simulated embeddings for each profile

embeddings = np.array([

    [0.1, 0.2, 0.3],  # Alice

    [0.3, 0.4, 0.5],  # Bob

    [0.5, 0.6, 0.7],  # Carol

], dtype=np.float32)

# Initialize FAISS index

def create_faiss_index(embeddings):

    dimension = embeddings.shape[1]

    index = faiss.IndexFlatL2(dimension)

    index.add(embeddings)

    print(f"FAISS index created with {index.ntotal} vectors.")

    return index

# Generate query embedding using OpenAI API

def generate_query_embedding(query):

    response = client.embeddings.create(

        input=query,

        model="text-embedding-ada-002"  # Keeping this model for course consistency

    )

    return np.array(response.data[0].embedding, dtype=np.float32)

# Retrieve top N profiles from FAISS

def retrieve_top_profiles(index, query_embedding, profiles, top_n=3):

    query_embedding = query_embedding.reshape(1, -1)

    distances, indices = index.search(query_embedding, top_n)

    results = [profiles[idx] for idx in indices[0]]

    return results

# Generate response using OpenAI GPT

def generate_response(query, retrieved_profiles):

    context = "\n".join([

        f"{profile['name']} is a {profile['role']} in {profile['department']}"

        for profile in retrieved_profiles

    ])

    prompt = f"Context: {context}\n\nQuery: {query}\n\nAnswer:"

    response = client.chat.completions.create(

        model="gpt-3.5-turbo",  # Updated from text-davinci-003

        messages=[

            {"role": "user", "content": prompt}

        ],

        max_tokens=100

    )

    return response.choices[0].message.content.strip()

# Main function

def main():

    # Step 1: Create FAISS index

    index = create_faiss_index(embeddings)

    # Step 2: Generate query embedding

    query = "Who can help me with Python development?"

    query_embedding = generate_query_embedding(query)

    # Step 3: Retrieve top profiles

    top_profiles = retrieve_top_profiles(index, query_embedding, profiles, top_n=3)

    print("Retrieved Profiles:", top_profiles)

    # Step 4: Generate response

    response = generate_response(query, top_profiles)

    print("Generated Response:", response)

# Execute script

if __name__ == "__main__":

    main()

Explanation of Each Step

Step 1: Create FAISS Index

• Initializes a FAISS index and adds embeddings to it for efficient similarity searches.

• Key Functions:

  • faiss.IndexFlatL2(dimension): Creates a basic FAISS index with L2 distance.

Step 2: Generate Query Embedding

• Converts a sample query into a vector using OpenAI’s embedding API.

• Key Functions:

  • openai.Embedding.create: Generates an embedding vector for the input text.

Step 3: Retrieve Top Profiles

• Searches the FAISS index for the N most similar embeddings to the query embedding.

• Maps retrieved indices back to the original profiles.

• Key Functions:

  • index.search(query_embedding, top_n): Performs the similarity search.

Step 4: Generate Response

• Builds a context from the retrieved profiles and passes it, along with the query, to the OpenAI API for generating a response.

• Key Functions:

  • openai.Completion.create: Generates a text-based response from the LLM.

Sample Output

1. Query: "Who can help me with Python development?"

2. Retrieved Profiles:

   • Alice: Python Developer in AI Development.

   • Bob: Data Scientist in Data Analytics.

3. Generated Response:

   • "Alice and Bob are experts in Python development. Alice specializes in AI applications, while Bob has extensive experience in data analytics."