Mastering Deployment of AI Pipelines 

Below you will find some suggested materials on these topics, which we believe may help you complete the final activity in the lesson. Please note that using these resources is entirely optional, and you are welcome to explore any other sources or courses at your own discretion.

Introduction (2 Minutes)

Opening Statement:

• • Welcome to today’s session on API Design Principles for AI Applications.

  • APIs serve as the backbone of modern AI systems, enabling seamless communication between applications and AI models like LLMs.

Agenda:

1. Overview of API design principles.

2. Structuring endpoints for LLM-based tasks.

3. Handling edge cases and errors in API responses.

Part 1: Overview of API Design Principles (5 Minutes)

What Makes a Well-Designed API?

• A well-designed API is intuitive, secure, and efficient, providing a clear interface for developers to interact with underlying functionalities.

Key Principles:

1. Consistency:

   • Maintain uniform naming conventions and data structures.

   • Example: Use camelCase or snake_case consistently for field names.

2. Scalability:

   • Design APIs to handle increasing loads, especially critical for AI applications.

   • Use pagination for large datasets and asynchronous processing for heavy computations.

3. Modularity:

   • Separate endpoints based on functionality (e.g., embedding generation, similarity search).

   • Example: /generate-embedding for creating embeddings and /search-similarity for querying.

4. Security:

   • Protect sensitive data with OAuth, API keys, and HTTPS.

   • Example: Validate user inputs to prevent SQL injection and other vulnerabilities.

5. Documentation:

   • Provide clear, up-to-date API documentation.

   • Example: Tools like Swagger and Postman make it easier for developers to test and understand your API.

Why These Principles Matter for AI APIs:

• AI tasks, such as querying LLMs or handling embeddings, often involve significant computational overhead. Following these principles ensures performance and usability.

Part 2: Structuring Endpoints for LLM-Based Tasks (5 Minutes)

Common LLM-Based Tasks:

1. Embedding Generation:

   • Converts text into numerical vectors.

   • Example Endpoint: POST /generate-embedding

2. Similarity Search:

   • Finds related data points in a vector database.

   • Example Endpoint: POST /search-similarity

3. Response Generation:

   • Uses an LLM to generate text based on inputs.

   • Example Endpoint: POST /generate-response

Structuring Endpoints:

• Use HTTP methods appropriately:

  • GET: Retrieve data (e.g., metadata or API status).

  • POST: Perform tasks that modify or generate data (e.g., generating embeddings).

  • PUT: Update existing records.

  • DELETE: Remove data.

Example API Design:

text

Copy code

POST /generate-embedding

{

  "text": "Who can help me with Python development?"

}

Response:

{

  "embedding": [0.12, 0.34, 0.56, ...]

}

text

Copy code

POST /search-similarity

{

  "embedding": [0.12, 0.34, 0.56, ...],

  "top_k": 3

}

Response:

{

  "results": [

    {"id": "1", "name": "Alice", "similarity": 0.95},

    {"id": "2", "name": "Bob", "similarity": 0.89}

  ]

}

text

Copy code

POST /generate-response

{

  "context": "Alice is a Python Developer skilled in AI.",

  "query": "Who can help with machine learning?"

}

Response:

{

  "response": "Alice is the best person for machine learning."

}

Key Considerations:

• Use versioning to maintain backward compatibility:

  • Example: /v1/generate-embedding

• Ensure endpoints are descriptive and self-explanatory.

Part 3: Handling Edge Cases and Errors in API Responses (5 Minutes)

Common Edge Cases:

1. Invalid Inputs:

   • Example: Missing fields or incorrect data types.

   • Solution: Validate inputs using schemas (e.g., JSON Schema).

2. No Results Found:

   • Example: A similarity search returns no matches.

Solution: Return an empty array with a clear message:
json
Copy code
{

  "results": [],

  "message": "No similar items found."

}

3. API Overload:

   • Example: Too many simultaneous requests.

   • Solution: Implement rate limiting and return a 429 Too Many Requests status code.

4. Timeouts:

   • Example: Long-running LLM queries.

   • Solution: Use asynchronous processing and provide status updates.

Error Handling Best Practices:

1. Use Standard HTTP Status Codes:

   • 200 OK: Success.

   • 400 Bad Request: Client-side input issues.

   • 500 Internal Server Error: Server-side problems.

2. Provide Helpful Error Messages:

   • Include details about the issue and possible solutions.

json
Copy code
{

  "error": "Invalid input",

  "details": "The 'text' field is required."

}

3. 
   

4. Log Errors Internally:

   • Use tools like ELK Stack or AWS CloudWatch to monitor and debug issues.

Example Error Response:

json

Copy code

{

  "status": 400,

  "error": "Invalid request",

  "message": "The 'embedding' field is missing."

}

Conclusion and Key Takeaways (3 Minutes)

Recap:

1. Design APIs with scalability, security, and modularity in mind.

2. Structure endpoints to handle core LLM tasks like embedding generation, similarity search, and response generation.

3. Anticipate and manage edge cases with robust validation and error handling.

Call to Action:

• Experiment with building and testing API endpoints for common AI tasks.

• Use tools like Postman or Swagger to document and debug your API.

• Continuously optimize for performance and usability.

Suggestions for Visual Aids:

• Slide 1: Title slide with “API Design Principles for AI Applications.”

• Slide 2: Diagram of a sample API workflow (client → embedding generation → FAISS → response generation).

• Slide 3: Example API requests and responses for LLM tasks.

• Slide 4: Table summarizing HTTP status codes and their uses.

Objective

In this hands-on activity, you will:

1. Design an API structure for the Jarvid RAG (Retrieval-Augmented Generation) pipeline.

2. Define routes for core tasks, including embedding generation, similarity retrieval, and response generation.

3. Ensure modularity and scalability for future extensions.

Step-by-Step Instructions

Step 1: Define API Endpoints

Overview of Tasks:

1. Embedding Generation: Converts input text into vector embeddings.

2. Retrieval: Searches the FAISS index for relevant embeddings.

3. Response Generation: Combines retrieved context and user queries to generate responses via an LLM API.

Endpoint Structure:

Endpoint

HTTP Method

Description

/generate-embedding

POST

Generates embeddings for input text.

/search-similarity

POST

Retrieves similar items from the FAISS index.

/generate-response

POST

Generates a response using retrieved context.

/health

GET

Checks API health and availability.

Step 2: Set Up the API Framework

Install FastAPI: FastAPI is a Python web framework for building APIs.
bash
Copy code
pip install fastapi uvicorn

Create the API Project Structure:
plaintext
Copy code

jarvid_api/
├── main.py         # Entry point for the API

├── routes/         # Directory for route files

│   ├── embeddings.py

│   ├── retrieval.py

│   └── responses.py

├── utils/          # Utility functions

│   ├── faiss_helper.py

│   └── openai_helper.py

└── requirements.txt

Step 3: Define Routes for Core Tasks

Embedding Generation

Route: /generate-embedding

Route Definition:

python

Copy code

from fastapi import APIRouter

from pydantic import BaseModel

import openai

router = APIRouter()

# Input schema

class EmbeddingRequest(BaseModel):

    text: str

# Route implementation

@router.post("/generate-embedding")

async def generate_embedding(request: EmbeddingRequest):

    openai.api_key = "your_openai_api_key"

    response = openai.Embedding.create(

        input=request.text,

        model="text-embedding-ada-002"

    )

    return {"embedding": response['data'][0]['embedding']}

Retrieval

Route: /search-similarity

Route Definition:

python

Copy code

from fastapi import APIRouter

from pydantic import BaseModel

import faiss

import numpy as np

router = APIRouter()

# Load FAISS index

index = faiss.read_index("vector_index.faiss ")

# Input schema

class RetrievalRequest(BaseModel):

    embedding: list

    top_k: int

# Route implementation

@router.post("/search-similarity")

async def search_similarity(request: RetrievalRequest):

    query_embedding = np.array(request.embedding, dtype=np.float32).reshape(1, -1)

    distances, indices = index.search(query_embedding, request.top_k)

    return {"distances": distances.tolist(), "indices": indices.tolist()}

Response Generation

Route: /generate-response

Route Definition:

python

Copy code

from fastapi import APIRouter

from pydantic import BaseModel

import openai

router = APIRouter()

# Input schema

class ResponseRequest(BaseModel):

    context: str

    query: str

# Route implementation

@router.post("/generate-response")

async def generate_response(request: ResponseRequest):

    openai.api_key = "your_openai_api_key"

    prompt = f"Context: {request.context}\n\nQuestion: {request.query}\n\nAnswer:"

    response = openai.Completion.create(

        engine="text-davinci-003",

        prompt=prompt,

        max_tokens=100

    )

    return {"response": response['choices'][0]['text'].strip()}

Health Check

Route: /health

Route Definition:

python

Copy code

from fastapi import APIRouter

router = APIRouter()

@router.get("/health")

async def health_check():

    return {"status": "API is running"}

Step 4: Combine Routes and Run the API

Combine All Routes:
python
Copy code

from fastapi import FastAPI

from routes.embeddings import router as embedding_router

from routes.retrieval import router as retrieval_router

from routes.responses import router as response_router

app = FastAPI()

# Include routes

app.include_router(embedding_router, prefix="/api/v1")

app.include_router(retrieval_router, prefix="/api/v1")

app.include_router(response_router, prefix="/api/v1")

@app.get("/")

def root():

    return {"message": "Welcome to the Jarvid API"}

Run the API: Use Uvicorn to launch the API server.
bash
Copy code
uvicorn main:app --reload

Step 5: Test the API

1. Test Endpoints:

   • Use Postman or Curl to test each endpoint.

Example Request for /generate-embedding:
json
Copy code
{

  "text": "Who can help me with Python development?"

}

Expected Response:
json
Copy code
{

    "embedding": [0.12, 0.34, 0.56, ...]

}

Step 6: Documentation

1. Generate API Documentation:

   • FastAPI automatically provides interactive API documentation.

   • Visit http://127.0.0.1:8000/docs to view and test the API.

2. Include Notes on Scalability and Error Handling:

   • Use rate limiting to prevent overloading.

   • Validate input data for each route.

   • Return informative error messages for invalid requests.

Expected Outcomes

1. Functional API Structure:

   • Routes for embedding generation, retrieval, and response generation are implemented and tested.

2. Interactive Documentation:

   • API documentation is auto-generated by FastAPI.

3. Scalability and Modularity:

   • The API design ensures easy future extensions and scalability.

1. Introduction

• Purpose: Define the API endpoints, request/response formats, and integration guidelines for the Jarvid pipeline.

• Audience: Developers, system integrators, and API consumers.

• Overview: The API provides functionalities for managing, executing, and monitoring tasks within the Jarvid pipeline.

2. API Overview

• Base URL:

  • Production: https://api.jarvid.com

  • Development: https://dev.api.jarvid.com

• Authentication:

  • Method: Bearer Token (OAuth2).

  • Endpoint: /auth/token

Example:
http
Copy code
Authorization: Bearer <token>

3. Endpoints

3.1. Pipeline Management

Endpoint: POST /pipelines

• Description: Create a new pipeline.

Request:
json
Copy code
{

  "name": "Pipeline Name",

  "description": "Pipeline description",

  "tasks": [

    {

      "type": "TaskType",

      "config": {

        "key": "value"

      }

    }

  ]

}

• 
  

• Response:

Success (201):
json
Copy code
{

  "pipeline_id": "12345",

  "status": "created"

}

Error (400/500):
json
Copy code
{

  "error": "Invalid payload"

}

Endpoint: GET /pipelines/{id}

• Description: Fetch details of a specific pipeline.

• Request: No body required.

Response:
json
Copy code
{

  "pipeline_id": "12345",

  "name": "Pipeline Name",

  "status": "active",

  "tasks": [...]

}

3.2. Task Execution

Endpoint: POST /tasks

• Description: Trigger execution of a task within a pipeline.

Request:
json
Copy code
{

  "pipeline_id": "12345",

  "task_id": "task-678",

  "parameters": {

    "key": "value"

  }

}

Response:
json
Copy code
{

  "task_id": "task-678",

  "status": "queued",

  "execution_id": "exec-789"

}

Endpoint: GET /tasks/{id}/status

• Description: Get the status of a task execution.

Response:
json
Copy code
{

  "execution_id": "exec-789",

  "status": "in_progress",

  "logs": ["Step 1 completed", "Step 2 running..."]

}

3.3. Monitoring

Endpoint: GET /pipelines/status

• Description: Fetch the status of all pipelines.

Response:
json
Copy code
[

  {

    "pipeline_id": "12345",

    "name": "Pipeline 1",

    "status": "active"

  },

  {

    "pipeline_id": "67890",

    "name": "Pipeline 2",

    "status": "failed"

  }

]

4. Error Handling

Standard Error Format:
json
Copy code
{

  "error": "Error message",

  "code": 400

}

• 
  

• Common Errors:

  • 400: Bad Request

  • 401: Unauthorized

  • 404: Not Found

  • 500: Internal Server Error

5. Rate Limiting

• Limits: 1000 requests per hour per token.

• Headers:

  • X-RateLimit-Limit: Total allowed requests.

  • X-RateLimit-Remaining: Requests remaining in the current window.

6. Examples

• Provide detailed cURL examples for each endpoint.

7. Versioning

• Current Version: v1

• Endpoint: /v1/{resource}