Troubleshooting Guide

🔧 Troubleshooting Guide

This guide helps you resolve common issues when setting up and using Insight Ingenious - an enterprise-grade Python library for AI agent APIs with Microsoft Azure integrations. The library includes comprehensive debugging utilities to help diagnose and resolve deployment issues.

� Quick Test Commands

Hello World Test (bike-insights)

# The "Hello World" of Ingenious - try this first!
# Note: Adjust port (80 or 8080) based on your server configuration
curl -X POST http://localhost:8080/api/v1/chat \
   -H "Content-Type: application/json" \
   -d '{
   "user_prompt": "{\"stores\": [{\"name\": \"Hello Store\", \"location\": \"NSW\", \"bike_sales\": [{\"product_code\": \"HELLO-001\", \"quantity_sold\": 1, \"sale_date\": \"2023-04-01\", \"year\": 2023, \"month\": \"April\", \"customer_review\": {\"rating\": 5.0, \"comment\": \"Perfect introduction!\"}}], \"bike_stock\": []}], \"revision_id\": \"hello-1\", \"identifier\": \"world\"}",
   "conversation_flow": "bike-insights"
   }'

Simple Alternative Test (classification-agent)

# If bike-insights seems too complex, try this simpler workflow
curl -X POST http://localhost:8080/api/v1/chat \
   -H "Content-Type: application/json" \
   -d '{
   "user_prompt": "Analyze this feedback: Great product!",
   "conversation_flow": "classification-agent"
   }'

---

�🚨 Common Setup Issues

1. Profile Validation Errors

Symptoms:

ValidationError: 9 validation errors for Profiles
0.chat_history.database_connection_string
  Input should be a valid string [type=string_type, input_value=None, input_type=NoneType]

Causes:

• Environment variables not set or empty
• Missing required fields in profiles.yml
• Incorrect environment variable syntax

Solutions:

1. Check your .env file:

   # Make sure .env exists and has these minimum variables
   cat .env
   

   Should contain:

   AZURE_OPENAI_API_KEY=your-actual-key
   AZURE_OPENAI_BASE_URL=https://your-endpoint.cognitiveservices.azure.com/
   

2. Use minimal profiles.yml:

   # Copy the minimal template
   cp ingenious/ingenious/ingenious_extensions_template/profiles.minimal.yml ./profiles.yml
   

3. Set environment variables:

   export INGENIOUS_PROJECT_PATH=$(pwd)/config.yml
   export INGENIOUS_PROFILE_PATH=$(pwd)/profiles.yml
   

---

2. Server Port Issues

Symptoms:

• “Permission denied” when starting server on port 80
• Server ignores --port parameter
• “Address already in use” errors
• Connection refused when testing API

Solutions:

1. Use alternative port (Recommended for development):

   # Use port 8080 instead of 80 (no admin privileges required)
   uv run ingen serve --port 8080
   
   # Update your test commands to use the new port
   curl http://localhost:8080/api/v1/health
   

2. Set port in environment variables:

   export WEB_PORT=8080
   uv run ingen serve
   

3. Or set in config.yml:

   web_configuration:
     port: 8080
   

4. Check if port is available:

   # Check what's using port 80
   lsof -i :80
   
   # Check what's using your target port
   lsof -i :8080
   
   # Kill processes if needed (be careful!)
   sudo kill -9 $(lsof -t -i:80)
   

5. For production deployments on port 80:

   # Run with elevated privileges (Linux/macOS)
   sudo uv run ingen serve
   
   # Or use a reverse proxy (nginx, apache)
   

Note: Port 80 requires administrative privileges on most systems. For development, use ports 8080, 8000, or 3000.

---

3. Module Import Errors

Symptoms:

ModuleNotFoundError: No module named 'ingenious_extensions'

Solutions:

1. Make sure you’re in the project root:

   pwd  # Should be your project directory
   ls   # Should see ingenious_extensions/ folder
   

2. Reinstall the library:

   uv add ingenious
   

3. Check Python path:

   uv run python -c "import sys; print('\n'.join(sys.path))"
   

---

4. Workflow Execution Errors

Symptoms:

• “Class ConversationFlow not found”
• “Expecting value: line 1 column 1 (char 0)”

Solutions:

1. Use correct workflow names:

   # ✅ Correct (preferred)
   "conversation_flow": "bike-insights"
   
   # ✅ Also supported (legacy)
   "conversation_flow": "bike_insights"
   

2. Check bike-insights input format:

   # bike-insights needs JSON in user_prompt
   curl -X POST http://localhost:80/api/v1/chat \
     -H "Content-Type: application/json" \
     -d '{
       "user_prompt": "{\"stores\": [...], \"revision_id\": \"test\", \"identifier\": \"test\"}",
       "conversation_flow": "bike-insights"
     }'
   

---

5. Azure SQL Database Issues

Symptoms:

• “pyodbc.InterfaceError: (‘IM002’, ‘[IM002] [Microsoft][ODBC Driver Manager] Data source name not found…”
• “Module pyodbc not found”
• Chat history not persisting between sessions
• Connection timeout errors

Prerequisites Check:

1. Verify ODBC Driver is installed:

   odbcinst -q -d
   # Should show: [ODBC Driver 18 for SQL Server]
   

2. Install ODBC Driver (if missing):

   On macOS:

   brew tap microsoft/mssql-release
   brew install msodbcsql18 mssql-tools18
   

   On Ubuntu/Debian:

   curl https://packages.microsoft.com/keys/microsoft.asc | apt-key add -
   curl https://packages.microsoft.com/config/ubuntu/20.04/prod.list > /etc/apt/sources.list.d/mssql-release.list
   apt-get update
   ACCEPT_EULA=Y apt-get install msodbcsql18
   

Configuration Solutions:

1. Install required dependencies:

   # Required for environment variable loading
   uv add python-dotenv
   

2. Check environment variable is set:

   echo $AZURE_SQL_CONNECTION_STRING
   # Should show your connection string
   
   # Or check if .env file is properly formatted
   cat .env | grep AZURE_SQL
   

3. Verify .env file format (critical):

   # .env file should have NO SPACES around = and quotes for complex values
   AZURE_SQL_CONNECTION_STRING="Driver={ODBC Driver 18 for SQL Server};Server=tcp:your-server.database.windows.net,1433;Database=your-database;Uid=your-username;Pwd=your-password;Encrypt=yes;TrustServerCertificate=no;Connection Timeout=30;"
   LOCAL_SQL_CSV_PATH=./sample_data.csv
   

4. Verify configuration files:

   # config.yml
   chat_history:
     database_type: "azuresql"
     database_name: "your_database_name"
   
   # profiles.yml
   chat_history:
     database_connection_string: ${AZURE_SQL_CONNECTION_STRING}
   

   Note: Remove :REQUIRED_SET_IN_ENV suffix - it can cause issues with environment variable substitution.

5. Test environment variable loading:

   uv run python -c "
   from dotenv import load_dotenv
   import os
   load_dotenv()
   conn_str = os.getenv('AZURE_SQL_CONNECTION_STRING')
   if not conn_str:
       print('❌ AZURE_SQL_CONNECTION_STRING not set')
   else:
       print('✅ Environment variable loaded successfully')
       print(f'Connection string length: {len(conn_str)} characters')
   "
   

6. Test connection directly:

   uv run python -c "
   import pyodbc
   import os
   conn_str = os.getenv('AZURE_SQL_CONNECTION_STRING')
   if not conn_str:
       print('❌ AZURE_SQL_CONNECTION_STRING not set')
   else:
       try:
           conn = pyodbc.connect(conn_str)
           print('✅ Azure SQL connection successful')
           conn.close()
       except Exception as e:
           print(f'❌ Connection failed: {e}')
   "
   

7. Test through Ingenious repository:

   uv run python -c "
   import asyncio
   from ingenious.ingenious.dependencies import get_config
   from ingenious.ingenious.db.chat_history_repository import ChatHistoryRepository
   from ingenious.models.database_client import DatabaseClientType
   
   async def test():
       config = get_config()
       db_type = DatabaseClientType(config.chat_history.database_type)
       repo = ChatHistoryRepository(db_type=db_type, config=config)
       try:
           messages = await repo.get_thread_messages('test-thread')
           print(f'✅ Azure SQL repository working! (Found {len(messages)} messages)')
       except Exception as e:
           print(f'❌ Repository error: {e}')
   
   asyncio.run(test())
   "
   

Common Connection String Issues:

• Missing driver: Ensure Driver={ODBC Driver 18 for SQL Server} is in the connection string
• Port issues: Use Server=tcp:your-server.database.windows.net,1433
• Encryption: Include Encrypt=yes;TrustServerCertificate=no
• Timeout: Add Connection Timeout=30 for slow networks

Security Notes:

• Never commit connection strings to version control
• Always use environment variables for database credentials
• Rotate passwords regularly for production deployments

---

6. Azure Blob Storage Issues

Symptoms:

• “BlobServiceClient cannot be constructed from connection string”
• “Storage account not found” or authentication errors
• Memory/prompts not persisting between sessions
• File operations failing silently
• “Container does not exist” errors

Prerequisites Check:

1. Verify Azure Storage SDK is installed:

   uv tree | grep azure-storage-blob
   # Should show: azure-storage-blob==12.24.0
   

2. Install Azure Storage SDK (if missing):

   uv add azure-storage-blob
   

Configuration Steps:

1. Set up Azure Storage Account (via Azure Portal):
   • Create a Storage Account (General Purpose v2)
   • Note the Account Name and Account Key
   • Get the Connection String from “Access keys” section
2. Configure environment variables:

   # Add to .env file
   AZURE_STORAGE_CONNECTION_STRING="DefaultEndpointsProtocol=https;AccountName=yourstorageaccount;AccountKey=your_key;EndpointSuffix=core.windows.net"
   AZURE_STORAGE_ACCOUNT_NAME="yourstorageaccount"
   AZURE_STORAGE_ACCOUNT_KEY="your_account_key"
   

3. Update configuration files:

   # config.yml
   storage:
     type: "azure_blob"
     container_data: "ingenious-data"
     container_revisions: "ingenious-revisions"
   
   # profiles.yml
   storage:
     type: "azure_blob"
     container_data: "ingenious-data-dev"  # Environment-specific
     container_revisions: "ingenious-revisions-dev"
   

4. Test environment variable loading:

   uv run python -c "
   from dotenv import load_dotenv
   import os
   load_dotenv()
   conn_str = os.getenv('AZURE_STORAGE_CONNECTION_STRING')
   if not conn_str:
       print('❌ AZURE_STORAGE_CONNECTION_STRING not set')
   else:
       print('✅ Environment variable loaded successfully')
       print(f'Connection string length: {len(conn_str)} characters')
   "
   

5. Test Azure Blob Storage connectivity:

   uv run python -c "
   from azure.storage.blob import BlobServiceClient
   import os
   conn_str = os.getenv('AZURE_STORAGE_CONNECTION_STRING')
   if not conn_str:
       print('❌ AZURE_STORAGE_CONNECTION_STRING not set')
   else:
       try:
           client = BlobServiceClient.from_connection_string(conn_str)
           account_info = client.get_account_information()
           print('✅ Azure Blob Storage connection successful')
           print(f'Account kind: {account_info[\"account_kind\"]}')
       except Exception as e:
           print(f'❌ Connection failed: {e}')
   "
   

6. Test through Ingenious FileStorage:

   uv run python -c "
   from ingenious.ingenious.files import get_file_storage
   from ingenious.ingenious.dependencies import get_config
   
   try:
       config = get_config()
       file_storage = get_file_storage()
       print(f'✅ FileStorage initialized: {type(file_storage).__name__}')
   
       # Test basic operations
       test_path = 'test/hello.txt'
       file_storage.save_text(test_path, 'Hello Azure!')
       content = file_storage.load_text(test_path)
       print(f'✅ File operations working: {content}')
   
       # Cleanup
       if file_storage.exists(test_path):
           file_storage.delete(test_path)
           print('✅ Cleanup successful')
   except Exception as e:
       print(f'❌ FileStorage error: {e}')
   "
   

7. Test Memory and Prompts Integration:

   uv run python -c "
   from ingenious.ingenious.services.memory_manager import MemoryManager
   import json
   
   try:
       memory_manager = MemoryManager()
       test_data = {'test': 'memory_data', 'timestamp': '2024-01-01'}
   
       # Test memory operations
       memory_manager.save_memory('test_conversation', test_data)
       loaded_data = memory_manager.load_memory('test_conversation')
       print(f'✅ Memory operations working: {loaded_data == test_data}')
   
       # Test prompts API
       import requests
       response = requests.get('http://localhost:8000/api/v1/prompts')
       if response.status_code == 200:
           print('✅ Prompts API accessible')
       else:
           print(f'⚠️  Prompts API returned: {response.status_code}')
   
   except Exception as e:
       print(f'❌ Memory/Prompts error: {e}')
   "
   

Common Connection String Issues:

• Malformed connection string: Ensure all required fields are present:

  DefaultEndpointsProtocol=https;AccountName=name;AccountKey=key;EndpointSuffix=core.windows.net
  

• Missing EndpointSuffix: Required for proper endpoint resolution
• Wrong account name/key: Verify credentials in Azure Portal
• Network access: Ensure storage account allows access from your IP/network

Container Management:

• Auto-creation: Ingenious automatically creates containers if they don’t exist
• Naming conventions: Use lowercase, hyphens only (e.g., ingenious-data-dev)
• Environment separation: Use different containers for dev/staging/prod

Performance Considerations:

• Connection reuse: BlobServiceClient instances are reused automatically
• Batch operations: For high-volume scenarios, consider batching operations
• Timeout settings: Adjust timeout in configuration for slow networks

Security Best Practices:

• Connection strings: Never commit to version control
• Access keys: Rotate regularly
• Network access: Configure firewall rules in Azure Portal
• Shared Access Signatures: Consider using SAS tokens for limited access

Troubleshooting Storage Type Conflicts:

If you’re switching from local to Azure Blob Storage:

1. Clear local data (if safe to do so):

   rm -rf ./data/conversations
   rm -rf ./data/prompts
   

2. Verify configuration precedence:
   • profiles.yml overrides config.yml
   • Environment variables override both
3. Test with minimal configuration:

   # Minimal profiles.yml for testing
   storage:
     type: "azure_blob"
     container_data: "test-data"
     container_revisions: "test-revisions"
   

---

🐛 Debugging Commands

Check System Status

uv run ingen status

List Available Workflows

uv run ingen workflows

Check Specific Workflow Requirements

uv run ingen workflows bike-insights

Test Installation

uv run python -c "import ingenious; print('✅ Ingenious imported successfully')"

Check Configuration Loading

export INGENIOUS_PROJECT_PATH=$(pwd)/config.yml
export INGENIOUS_PROFILE_PATH=$(pwd)/profiles.yml
uv run python -c "
import ingenious.config.config as config
try:
    cfg = config.get_config()
    print('✅ Configuration loaded successfully')
    print(f'Models: {len(cfg.models)}')
    print(f'Profile: {cfg.chat_history.database_type}')
except Exception as e:
    print(f'❌ Configuration error: {e}')
"

---

🔍 Log Analysis

Enable Debug Logging

1. In config.yml:

   logging:
     root_log_level: debug
     log_level: debug
   

2. Or via environment:

   export LOGLEVEL=DEBUG
   export ROOTLOGLEVEL=DEBUG
   

Common Log Messages

✅ Good Signs:

Profile loaded from file
Module ingenious_extensions.services.chat_services.multi_agent.conversation_flows.bike_insights.bike_insights found.
DEBUG: Successfully loaded conversation flow class
INFO:     Uvicorn running on http://0.0.0.0:80

⚠️ Warning Signs:

Profile not found at /path/to/profiles.yml
Template directory not found. Skipping...
Validation error in field

❌ Error Signs:

ModuleNotFoundError: No module named
ValidationError: 9 validation errors
Class ConversationFlow not found in module

---

🧪 Testing & Verification

Minimal Test

# Test server is running
curl -s http://localhost:80/api/v1/health || echo "Server not responding"

# Test bike-insights workflow
curl -X POST http://localhost:80/api/v1/chat \
  -H "Content-Type: application/json" \
  -d '{
    "user_prompt": "{\"stores\": [], \"revision_id\": \"test\", \"identifier\": \"test\"}",
    "conversation_flow": "bike-insights"
  }' | jq '.message_id // "ERROR"'

Full Integration Test

#!/bin/bash
set -e

echo "🧪 Running full integration test..."

# 1. Check environment
echo "1. Checking environment..."
[ -n "$AZURE_OPENAI_API_KEY" ] || { echo "❌ AZURE_OPENAI_API_KEY not set"; exit 1; }
[ -f "config.yml" ] || { echo "❌ config.yml not found"; exit 1; }
[ -f "profiles.yml" ] || { echo "❌ profiles.yml not found"; exit 1; }

# 2. Test import
echo "2. Testing Python import..."
uv run python -c "import ingenious; print('✅ Import OK')"

# 3. Test configuration
echo "3. Testing configuration..."
export INGENIOUS_PROJECT_PATH=$(pwd)/config.yml
export INGENIOUS_PROFILE_PATH=$(pwd)/profiles.yml
uv run ingen status

# 4. Test workflows
echo "4. Testing workflows..."
uv run ingen workflows | grep -q "bike-insights" && echo "✅ bike-insights available"

echo "✅ All tests passed!"

---

📋 Environment Checklist

Before running Ingenious, ensure:

• Python 3.13+ installed
• uv package manager available
• Ingenious library installed: uv add ingenious
• Project initialized: uv run ingen init
• .env file created with Azure OpenAI credentials
• Environment variables set:
  • AZURE_OPENAI_API_KEY
  • AZURE_OPENAI_BASE_URL
  • INGENIOUS_PROJECT_PATH
  • INGENIOUS_PROFILE_PATH
• Port available (default 80)
• Network access to Azure OpenAI endpoint

---

🆘 Getting Help

Self-Help Commands

# Get general help
uv run ingen --help

# Get command-specific help
uv run ingen serve --help
uv run ingen workflows --help

# Check system status
uv run ingen status

# List all workflows
uv run ingen workflows

Common Solutions Quick Reference

Issue	Quick Fix
Import errors	uv add ingenious
Profile validation	Use profiles.minimal.yml template
Port not working	Set WEB_PORT environment variable
Workflow not found	Use bike-insights (preferred) or bike_insights (legacy)
JSON parse error	Escape quotes in user_prompt for bike-insights
Server won’t start	Check port availability and config.yml

Still Need Help?

1. Check the logs for specific error messages
2. Review configuration files against templates
3. Test with minimal configuration first
4. Check the API documentation: /docs/api/WORKFLOWS.md
5. Verify environment variables are loaded correctly

---

🔄 Reset Instructions

If everything is broken, start fresh:

# 1. Clean slate
rm -rf ingenious_extensions/ tmp/ config.yml profiles.yml .env

# 2. Reinstall
uv add ingenious

# 3. Initialize
uv run ingen init

# 4. Configure
cp .env.example .env
# Edit .env with your Azure OpenAI credentials

# 5. Set environment
export INGENIOUS_PROJECT_PATH=$(pwd)/config.yml
export INGENIOUS_PROFILE_PATH=$(pwd)/profiles.yml

# 6. Test
uv run ingen status
uv run ingen serve