Sugar-AI Project

This document describes how to set up Sugar-AI, run it locally, and troubleshoot common issues.

Quick start

The default contributor setup uses Python and CPU-only PyTorch. A CUDA profile and the existing CUDA 12.8 Docker image are available for NVIDIA systems.

1. Clone the repository and create a virtual environment

From the repository root, run:

python -m venv .venv
.venv\Scripts\Activate.ps1
python -m pip install --upgrade pip

On Linux or macOS, activate the environment with source .venv/bin/activate instead.

2. Create the active environment file

.example.env is the repository template. Copy it to .env and put local secrets and configuration in .env; do not commit .env.

Copy-Item .example.env .env

On Linux or macOS:

cp .example.env .env

The template is configured for development mode with the small Hugging Face model HuggingFaceTB/SmolLM2-135M-Instruct and the documentation files included in this repository. The first start downloads the language and embedding models and builds the document index, so it can take several minutes.

3. Install dependencies

For the default CPU setup:

python -m pip install -r requirements.txt

requirements.txt selects the CPU profile. On a supported NVIDIA CUDA system, choose the CUDA profile explicitly instead:

python -m pip install -r requirements/cuda.txt

The CUDA profile requires a compatible NVIDIA driver and installs the CUDA-enabled PyTorch and bitsandbytes packages. Do not install both profiles into the same virtual environment; create a fresh environment when switching profiles.

4. Start the server

python -m uvicorn main:app --host 0.0.0.0 --port 8000

The API is available at http://localhost:8000/. A successful startup ends with Application startup complete. Check the provider and model with:

curl http://localhost:8000/health

Environment and model selection

Sugar-AI reads the active configuration from .env. The model is selected in this order:

AI_MODEL, when set.
DEV_MODEL_NAME, when DEV_MODE=1.
PROD_MODEL_NAME, when DEV_MODE=0.

AI_PROVIDER must match the model service being used. The template uses huggingface. For Ollama, OpenAI-compatible services, or Gemini, uncomment and configure the corresponding provider settings in .env.

The application fails during startup when no model is configured. This gives a clear configuration error instead of passing an empty model name to a provider.

Running Sugar-AI with Docker

The Dockerfile preserves the existing CUDA 12.8 build and runtime images. Use this path on a host with Docker and, for GPU execution, the NVIDIA Container Toolkit. CPU contributors should use the Python CPU profile above because the Docker image still includes the CUDA runtime.

Build the image from the repository root:

docker build -t sugar-ai:local .

Run it with the active environment file and expose the API on port 8000:

docker run --gpus all --rm --env-file .env -p 8000:8000 sugar-ai:local

If the host has no NVIDIA GPU, omit --gpus all; the image can start with CPU execution, but it still requires the larger CUDA image and is not the recommended CPU setup.

Alternatively, use Docker Compose. Compose reads .env at runtime and does not bake it into the image:

docker compose up --build

The existing Compose configuration exposes the API at http://localhost:443/.

The Docker build downloads dependency wheels with retries and installs them offline in the image. If a build fails while downloading a large package, run the same build again; a transient or truncated package download should not be treated as a source-code failure.

Local Development (DEV_MODE)

Development mode uses the lightweight model from DEV_MODEL_NAME, which is suitable for testing on lower-memory machines.

Set these values in .env before starting the server:

DEV_MODE=1
DEV_MODEL_NAME=HuggingFaceTB/SmolLM2-135M-Instruct

Testing the FastAPI App

The FastAPI server provides endpoints to interact with Sugar-AI.

Test API endpoints

Sugar-AI provides three different endpoints for different use cases:

Endpoint	Purpose	Input Format	Features
`/ask`	RAG-enabled answers	Query parameter	• Retrieval-Augmented Generation • Sugar/Pygame/GTK documentation • Child-friendly responses
`/ask-llm`	Direct LLM without RAG	Query parameter	• No document retrieval • Direct model access • Faster responses • Default system prompt and parameters
`/ask-llm-prompted(promoted mode[default])`	Custom prompt with advanced controls	JSON body	• Custom system prompts • Configurable model parameters
`/ask-llm-prompted(chat=True)`	Accepts chat history with system prompt	JSON body	• Send chat history along with system prompt • Configurable model parameters

GET endpoint

Access the root URL:
http://localhost:8000/ to see the welcome message.
POST endpoint for asking questions

To submit a coding question, send a POST request to /ask with the question parameter. For example:
```
curl -X POST "http://localhost:8000/ask?question=How%20do%20I%20create%20a%20Pygame%20window?"
```
The API returns a JSON object with the answer.
POST endpoint for debugging python programs

To submit your code, send a POST request to /debug with the code parameter and a context flag. For example:
```
curl -X POST "http://localhost:8000/debug?code=How%20do%20I%20create%20a%20Pygame%20window&context=False?"
```
The API returns a JSON object with the answer.
Additional POST endpoint (/ask-llm)

An alternative endpoint /ask-llm is available in main.py, which provides similar functionality with an enhanced processing pipeline for LLM interactions. To use it, send your coding-related question using:
```
curl -X POST "http://localhost:8000/ask-llm?question=How%20do%20I%20create%20a%20Pygame%20window?"
```
The response format is JSON containing the answer generated by the language model.

Advanced POST endpoint - Custom prompt OR Chat completions + generation parameters (/ask-llm-prompted)

A powerful endpoint that allows you to use custom prompts and fine-tune generation parameters. Unlike the other endpoints, this one:

Uses your own custom system prompt
Accepts JSON request body with configurable model parameters
Provides direct LLM access without RAG

Basic Usage (Prompted mode):

curl -X POST "http://localhost:8000/ask-llm-prompted" \
  -H "X-API-Key: sugarai2024" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "How do I create a Pygame window?",
    "custom_prompt": "You are a Python expert. Provide detailed code examples with explanations."
  }'

Advanced Usage with Generation Parameters (Prompted mode):

curl -X POST "http://localhost:8000/ask-llm-prompted" \
  -H "X-API-Key: sugarai2024" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Write a function to calculate fibonacci numbers",
    "custom_prompt": "You are a coding tutor. Explain step-by-step with comments.",
    "max_length": 1024,
    "truncation": true,
    "repetition_penalty": 1.1,
    "temperature": 0.7,
    "top_p": 0.9,
    "top_k": 50,
    "think": false
  }'

Chat Completions Usage (chat mode):

If you want to send a conversation history and generate a model response then use this endpoint. The structure of the request and response is kept similar to any popular API providers. Use the same endpoint with chat: true and provide messages instead of question/custom_prompt.

curl -X POST "http://localhost:8000/ask-llm-prompted" \
  -H "X-API-Key: sugarai2024" \
  -H "Content-Type: application/json" \
  -d '{
    "chat": true,
    "messages": [
      {"role": "system", "content": "You are a helpful Python assistant."},
      {"role": "user", "content": "Write a Python function to reverse a string."},
      {"role": "assistant", "content": "You can use slicing: s[::-1]."},
      {"role": "user", "content": "Show me a complete example."}
    ],
    "max_length": 512,
    "temperature": 0.6,
    "top_p": 0.9,
    "top_k": 50,
    "think": false
  }'

Send chat history with roles system, user, and assistant.
The response mirrors common chat APIs and includes choices[0].message.content.

Request Parameters:

chat (optional, default: false): When true, use chat completions mode.
question (required when chat=false): The question or task to process
custom_prompt (required when chat=false): Your custom system prompt
messages (required when chat=true): Array of {role, content} messages where role is one of system, user, assistant
max_length (optional, default: 1024): Maximum length of generated response
truncation (optional, default: true): Whether to truncate long inputs
repetition_penalty (optional, default: 1.1): Controls repetition (1.0 = no penalty, >1.0 = less repetition)
temperature (optional, default: 0.7): Controls randomness (0.0 = deterministic, 1.0 = very random)
top_p (optional, default: 0.9): Nucleus sampling (0.1 = focused, 0.9 = diverse)
top_k (optional, default: 50): Limits vocabulary to K most likely words
think (optional, default: false): Enable model reasoning for this request. See Reasoning (think / no-think) below.

Response Format (Prompted mode):

{
  "answer": "Here's how to create a Pygame window:\n\nimport pygame...",
  "user": "Admin Key",
  "quota": {"remaining": 95, "total": 100},
  "generation_params": {
    "max_length": 1024,
    "truncation": true,
    "repetition_penalty": 1.1,
    "temperature": 0.7,
    "top_p": 0.9,
    "top_k": 50,
    "think": false
  }
}

Response Format (Chat mode):

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "Here is a complete example..."
      },
      "index": 0,
      "finish_reason": "stop"
    }
  ],
  "user": "Admin Key",
  "quota": {"remaining": 95, "total": 100},
  "generation_params": {
    "max_length": 512,
    "truncation": true,
    "repetition_penalty": 1.1,
    "temperature": 0.6,
    "top_p": 0.9,
    "top_k": 50,
    "think": false
  }
}

Actual response of the model can be accessed by:

data = json.loads(response)
content = data["choices"][0]["message"]["content"]
print(content)

Use Cases: Prompted Mode: Different activites can now use different system prompts and different generation parameters to achieve a model that is personalized to that activites needs.
Chat Mode: They can also use the chat mode to give context of chat history to the LLM better suited for conversational style features.

Generation Parameter Guidelines:

For Code: temperature: 0.2-0.4, top_p: 0.8, repetition_penalty: 1.1
For Creative Content: temperature: 0.7-0.9, top_p: 0.9, repetition_penalty: 1.2
For Factual Answers: temperature: 0.3-0.5, top_p: 0.7, repetition_penalty: 1.0

Reasoning (think / no-think)

Reasoning-capable models can spend extra tokens on reasoning before answering. This can help multi-step tasks, but it costs extra latency and tokens. Sugar-AI keeps reasoning off by default on every endpoint and exposes it as an opt-in think flag.

think is accepted on all generation endpoints. On /ask, /ask-llm, and /debug, it is a query flag. On /ask-llm-prompted, it goes in the JSON body alongside the other generation parameters. If callers omit it, the endpoint behaves as before with no-think.

# simple endpoint: think is a query flag
curl -X POST "http://localhost:8000/ask-llm?question=Explain%20recursion&think=true" \
  -H "X-API-Key: sugarai2024"

# prompted endpoint: think goes in the JSON body
curl -X POST "http://localhost:8000/ask-llm-prompted" \
  -H "X-API-Key: sugarai2024" \
  -H "Content-Type: application/json" \
  -d '{
    "question": "Refactor this recursive function to be iterative and explain the tradeoffs.",
    "custom_prompt": "You are a senior Python mentor.",
    "think": true,
    "max_length": 1024
  }'

Things to know:

Default is no-think everywhere. Existing callers that omit think are unaffected.
On /ask and /debug, think=true applies to the first answer or analysis stage only. The child-friendly rewrite stage always runs no-think.
Ollama is the only provider that honors think today. The OpenAI-compatible base provider, Gemini, and HuggingFace currently ignore it.
Some Ollama models reject the think field. Sugar-AI retries once without think so those models can still return an answer.
Model behavior is capability dependent. Pure reasoning or reasoning-style models may still include reasoning text even when think=false.
Reasoning shares the output token budget with the answer, so Sugar-AI adds THINKING_HEADROOM tokens when think=true. The default is 2048 and can be changed in .env.

API Authentication

Sugar-AI implements an API key-based authentication system for secure access to endpoints.

Setting Up Authentication

API keys are defined in the .env file with the following format:

API_KEYS={"sugarai2024": {"name": "Admin Key", "can_change_model": true}, "user_key_1": {"name": "User 1", "can_change_model": false}}

Each key has associated user information:

name: A friendly name for the user (appears in API responses and logs)
can_change_model: Boolean that controls permission to change the model

Testing Authentication

To use the authenticated endpoints, include the API key in your request headers:

curl -X POST "http://localhost:8000/ask?question=How%20do%20I%20create%20a%20Pygame%20window?" \
  -H "X-API-Key: sugarai2024"

The response will include the user name:

{
  "answer": "To create a Pygame window...",
  "user": "Admin Key"
}

Changing Models (Admin Only)

Users with can_change_model: true permission can change the model:

curl -X POST "http://localhost:8000/change-model?model=Qwen/Qwen2-1.5B-Instruct&api_key=sugarai2024&password=sugarai2024"

Why User Names Are Useful

The user name serves several purposes:

It provides identification in API responses, helping track which user made which request
It adds context to server logs for monitoring API usage
It allows for more personalized interaction in multi-user environments
It helps administrators identify which API key corresponds to which user

Advanced Security Features

Sugar-AI includes several additional security features to protect the API and manage resources effectively:

Request Quotas

Each API key has a daily request limit defined in the .env file:

MAX_DAILY_REQUESTS=100

The system automatically tracks usage and resets quotas daily. When testing:

Check remaining quota by examining API responses:

{
  "answer": "Your answer here...",
  "user": "User 1",
  "quota": {"remaining": 95, "total": 100}
}

Test quota enforcement by sending more than the allowed number of requests. The API will return a 429 status code when the quota is exceeded:

curl -i -X POST "http://localhost:8000/ask?question=Test" -H "X-API-Key: user_key_1"
# After exceeding quota:
# HTTP/1.1 429 Too Many Requests
# {"detail":"Daily request quota exceeded"}

Security Logging

Sugar-AI implements comprehensive logging for security monitoring:

All API requests are logged with user information, IP addresses, and timestamps
Failed authentication attempts are recorded with warning level
Model change attempts are tracked with detailed information
All logs are stored in sugar_ai.log for review

To test logging functionality:

# Make a valid request
curl -X POST "http://localhost:8000/ask?question=Test" -H "X-API-Key: sugarai2024"

# Make an invalid request
curl -X POST "http://localhost:8000/ask?question=Test" -H "X-API-Key: invalid_key"

# Check the logs
tail -f sugar_ai.log

CORS and Trusted Hosts

The API implements CORS (Cross-Origin Resource Sharing) and trusted host verification:

In development mode, API access is allowed from all origins
For production, consider restricting the allow_origins parameter in main.py

Testing with Streamlit App

The Streamlit app should be updated to include API key authentication and support for all three endpoints:

# Updated streamlit.py example
import streamlit as st
import requests
import json

st.title("Sugar-AI Chat Interface")

# Add API key field
api_key = st.sidebar.text_input("API Key", type="password")

# Endpoint selection
endpoint_choice = st.selectbox(
    "Choose endpoint:",
    ["RAG (ask)", "Direct LLM (ask-llm)", "Custom Prompt (ask-llm-prompted)"]
)

st.subheader("Ask Sugar-AI")
question = st.text_input("Enter your question:")

# Custom prompt section for ask-llm-prompted
custom_prompt = ""
generation_params = {}

if endpoint_choice == "Custom Prompt (ask-llm-prompted)":
    custom_prompt = st.text_area(
        "Custom Prompt:", 
        value="You are a helpful assistant. Provide clear and detailed answers.",
        help="This prompt will replace the default system prompt"
    )
    
    # Generation parameters
    with st.expander("Advanced Generation Parameters"):
        col1, col2 = st.columns(2)
        
        with col1:
            max_length = st.number_input("Max Length", value=1024, min_value=100, max_value=2048)
            temperature = st.slider("Temperature", 0.0, 1.0, 0.7, 0.1)
            repetition_penalty = st.slider("Repetition Penalty", 0.5, 2.0, 1.1, 0.1)
        
        with col2:
            top_p = st.slider("Top P", 0.1, 1.0, 0.9, 0.1)
            top_k = st.number_input("Top K", value=50, min_value=1, max_value=100)
            truncation = st.checkbox("Truncation", value=True)
    
    generation_params = {
        "max_length": max_length,
        "truncation": truncation,
        "repetition_penalty": repetition_penalty,
        "temperature": temperature,
        "top_p": top_p,
        "top_k": top_k
    }

if st.button("Submit"):
    if question and api_key:
        headers = {"X-API-Key": api_key}
        
        try:
            if endpoint_choice == "RAG (ask)":
                url = "http://localhost:8000/ask"
                params = {"question": question}
                response = requests.post(url, params=params, headers=headers)
                
            elif endpoint_choice == "Direct LLM (ask-llm)":
                url = "http://localhost:8000/ask-llm"
                params = {"question": question}
                response = requests.post(url, params=params, headers=headers)
                
            elif endpoint_choice == "Custom Prompt (ask-llm-prompted)":
                url = "http://localhost:8000/ask-llm-prompted"
                headers["Content-Type"] = "application/json"
                data = {
                    "question": question,
                    "custom_prompt": custom_prompt,
                    **generation_params
                }
                response = requests.post(url, headers=headers, data=json.dumps(data))
            
            if response.status_code == 200:
                result = response.json()
                st.markdown("**Answer:** " + result["answer"])
                st.sidebar.info(f"User: {result.get('user', 'Unknown')}")
                st.sidebar.info(f"Remaining quota: {result['quota']['remaining']}/{result['quota']['total']}")
                
                # Show generation parameters for custom prompt endpoint
                if endpoint_choice == "Custom Prompt (ask-llm-prompted)" and "generation_params" in result:
                    with st.expander("Generation Parameters Used"):
                        st.json(result["generation_params"])
                        
            else:
                st.error(f"Error {response.status_code}: {response.text}")
                
        except Exception as e:
            st.error(f"Error contacting the API: {e}")
            
    elif not question:
        st.warning("Please enter a question.")
    elif not api_key:
        st.warning("Please enter an API key.")

Run this updated Streamlit app to test the complete authentication flow and quota visibility.

Running the RAG Agent from the Command Line

To test the new RAG Agent directly from the CLI, execute:

python rag_agent.py --quantize

Remove the --quantize flag if you prefer running without 4‑bit quantization.

Testing the New Features

Verify Model Setup:
- Confirm the selected model loads correctly by checking the terminal output for any errors.
Document Retrieval:
- Place your documents (PDF or text files) in the directory specified in the default parameters or provide your paths using the --docs flag.
- The vector store is rebuilt every time the agent starts. Ensure your documents are well placed to retrieve relevant content.
Question Handling:
- After the agent starts, enter a sample coding-related question.
- The assistant should respond by incorporating context from the loaded documents and answering your query.
API and Docker Route:
- Optionally, combine these changes by deploying the updated version via Docker and testing the FastAPI endpoints as described above.

Troubleshooting CUDA Memory Issues

If you encounter CUDA out-of-memory errors, consider running the agent on CPU or adjust CUDA settings:

export PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True

Review the terminal output for further details and error messages.

Setting up CI/CD Environment Variables

When deploying Sugar-AI in CI/CD pipelines, you'll need to configure environment variables properly. Current CI/CD uses github webhooks. So make sure to create a webhook secret and add it to the .env.

Using the Streamlit App

Sugar-AI also provides a Streamlit-based interface for quick interactions and visualizations.

Running the Streamlit App

Install Streamlit:

If you haven't already, install Streamlit:
```
pip install streamlit
```

Make sure server is running using:

uvicorn main:app --host 0.0.0.0 --port 8000

Start the App:

Launch the Streamlit app by adding streamlit.py file.

#./streamlit.py
import streamlit as st
import requests

st.title("Sugar-AI Chat Interface")

use_rag = st.checkbox("Use RAG (Retrieval-Augmented Generation)", value=True)

st.subheader("Ask Sugar-AI")
question = st.text_input("Enter your question:")

if st.button("Submit"):
    if question:
        if use_rag:
            url = "http://localhost:8000/ask"
        else:
            url = "http://localhost:8000/ask-llm"
        params = {"question": question}
        try:
            response = requests.post(url, params=params)
            if response.status_code == 200:
                result = response.json()
                st.markdown("**Answer:** " + result["answer"])
            else:
                st.error(f"Error {response.status_code}: {response.text}")
        except Exception as e:
            st.error(f"Error contacting the API: {e}")
    else:
        st.warning("Please enter a question.")

streamlit run streamlit.py

Using the App:
- The app provides a simple UI to input coding questions and displays the response using Sugar-AI.
- Use the sidebar options to configure settings if available.
- The app communicates with the FastAPI backend to process and retrieve answers.

Enjoy exploring Sugar-AI through both API endpoints and the interactive Streamlit interface!

Name		Name	Last commit message	Last commit date
.github		.github
app		app
docs		docs
requirements		requirements
static		static
templates		templates
tests		tests
.example.env		.example.env
.gitignore		.gitignore
COPYING		COPYING
Dockerfile		Dockerfile
README.md		README.md
docker-compose.yaml		docker-compose.yaml
main.py		main.py
pytest.ini		pytest.ini
requirements-dev.txt		requirements-dev.txt
requirements.txt		requirements.txt

Navigation Menu

Repository navigation

About

Repository files navigation

Sugar-AI Project

Quick start

1. Clone the repository and create a virtual environment

2. Create the active environment file

3. Install dependencies

4. Start the server

Environment and model selection

Running Sugar-AI with Docker

Local Development (DEV_MODE)

Testing the FastAPI App

Test API endpoints

Reasoning (think / no-think)

API Authentication

Setting Up Authentication

Testing Authentication

Changing Models (Admin Only)

Why User Names Are Useful

Advanced Security Features

Request Quotas

Security Logging

CORS and Trusted Hosts

Testing with Streamlit App

Running the RAG Agent from the Command Line

Testing the New Features

Troubleshooting CUDA Memory Issues

Setting up CI/CD Environment Variables

Using the Streamlit App

Running the Streamlit App

About

Releases

Packages

Used by

Contributors

Languages

Latest commit

History

Folders and files

Repository files navigation

Sugar-AI Project

Quick start

1. Clone the repository and create a virtual environment

2. Create the active environment file

3. Install dependencies

4. Start the server

Environment and model selection

Running Sugar-AI with Docker

Local Development (DEV_MODE)

Testing the FastAPI App

Test API endpoints

Reasoning (think / no-think)

API Authentication

Setting Up Authentication

Testing Authentication

Changing Models (Admin Only)

Why User Names Are Useful

Advanced Security Features

Request Quotas

Security Logging

CORS and Trusted Hosts

Testing with Streamlit App

Running the RAG Agent from the Command Line

Testing the New Features

Troubleshooting CUDA Memory Issues

Setting up CI/CD Environment Variables

Using the Streamlit App

Running the Streamlit App

About

Resources

Stars

Watchers

Forks

Releases

Packages

Used by

Contributors

Languages