Output Schemas

Define JSON schemas to guarantee structured output from your models and agents.

What are Output Schemas?

Output schemas enforce a specific JSON structure on LLM responses. Instead of free-text, you get reliable, parseable data.

Without schema:

The sentiment is positive with high confidence. I detected 
entities: Apple (company), iPhone (product).

With schema:

{
  "sentiment": "positive",
  "confidence": 0.95,
  "entities": [
    {"text": "Apple", "type": "company"},
    {"text": "iPhone", "type": "product"}
  ]
}

Benefits

Reliable Parsing - Always get valid JSON
Type Safety - Enforce data types (string, number, boolean)
Validation - Enum values, ranges, required fields
Integration - Easy to integrate with downstream systems
No Parsing Errors - Never fails to extract structured data

Creating Schemas

Via UI

Go to Schemas
Click Create Schema
Name your schema (e.g., sentiment-analysis)
Define JSON structure
Save

Schema Format

Schemas use JSON Schema format:

{
  "type": "object",
  "properties": {
    "field_name": {
      "type": "string|number|boolean|array|object",
      "description": "What this field contains"
    }
  },
  "required": ["field_name"]
}

Common Schema Examples

Sentiment Analysis

{
  "type": "object",
  "properties": {
    "sentiment": {
      "type": "string",
      "enum": ["positive", "negative", "neutral"],
      "description": "Overall sentiment"
    },
    "confidence": {
      "type": "number",
      "minimum": 0,
      "maximum": 1,
      "description": "Confidence score (0-1)"
    },
    "reasoning": {
      "type": "string",
      "description": "Why this sentiment was chosen"
    }
  },
  "required": ["sentiment", "confidence"]
}

Entity Extraction

{
  "type": "object",
  "properties": {
    "entities": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "text": {"type": "string"},
          "type": {
            "type": "string",
            "enum": ["person", "company", "location", "product"]
          },
          "confidence": {"type": "number"}
        },
        "required": ["text", "type"]
      }
    }
  },
  "required": ["entities"]
}

Classification

{
  "type": "object",
  "properties": {
    "category": {
      "type": "string",
      "description": "Primary category"
    },
    "subcategory": {
      "type": "string",
      "description": "Specific subcategory"
    },
    "confidence": {
      "type": "number",
      "minimum": 0,
      "maximum": 1
    },
    "tags": {
      "type": "array",
      "items": {"type": "string"}
    }
  },
  "required": ["category", "confidence"]
}

Product Review

{
  "type": "object",
  "properties": {
    "rating": {
      "type": "integer",
      "minimum": 1,
      "maximum": 5
    },
    "pros": {
      "type": "array",
      "items": {"type": "string"}
    },
    "cons": {
      "type": "array",
      "items": {"type": "string"}
    },
    "recommendation": {
      "type": "boolean",
      "description": "Would recommend this product"
    },
    "summary": {
      "type": "string",
      "description": "Brief summary of review"
    }
  },
  "required": ["rating", "recommendation", "summary"]
}

Q&A with Citations

{
  "type": "object",
  "properties": {
    "answer": {
      "type": "string",
      "description": "The answer to the question"
    },
    "confidence": {
      "type": "string",
      "enum": ["high", "medium", "low"]
    },
    "citations": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "source": {"type": "string"},
          "quote": {"type": "string"}
        }
      }
    }
  },
  "required": ["answer", "confidence"]
}

Data Validation

{
  "type": "object",
  "properties": {
    "is_valid": {
      "type": "boolean",
      "description": "Whether data is valid"
    },
    "errors": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "field": {"type": "string"},
          "issue": {"type": "string"}
        }
      }
    },
    "suggestions": {
      "type": "array",
      "items": {"type": "string"}
    }
  },
  "required": ["is_valid"]
}

Using Schemas

With Adapters

Link a schema to your adapter:

Go to adapter settings
Select Output Schema
Choose your schema
Save

All inference with this adapter now returns structured data:

response = requests.post(
    "https://mlstack.io/api/llm/predict",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "adapter_name": "sentiment-classifier",
        "prompt": "This product is amazing!"
    }
)

# Response automatically follows schema
result = poll_result(job_id)
print(result["output"])
# {"sentiment": "positive", "confidence": 0.92}

With Base Models

Use schema_name parameter:

response = requests.post(
    "https://mlstack.io/api/llm/predict",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "base_model": "meta-llama/Llama-3.1-8B-Instruct",
        "schema_name": "sentiment-analysis",
        "prompt": "This product is terrible"
    }
)

Override Adapter Schema

Use different schema than adapter's default:

{
    "adapter_name": "my-classifier",
    "schema_name": "different-schema",  # Overrides adapter's schema
    "prompt": "..."
}

Disable Structured Output

Set structured: false:

{
    "adapter_name": "my-adapter",
    "structured": false,  # Get free-text instead
    "prompt": "..."
}

With Agents

Agents can use schemas in two ways:

1. Explicit Schema (Recommended for Complex Formats):

Create agent → Select output schema → Ensures agent's final response follows format

Important: Agent schemas must include: - final_answer in the action enum - answer field (type: string) for the final response

{
  "type": "object",
  "properties": {
    "action": {
      "type": "string",
      "enum": ["tool1", "tool2", "final_answer"]
    },
    "tool1_param": {"type": "string"},
    "tool2_param": {"type": "string"},
    "answer": {
      "type": "string",
      "description": "Final answer when action is final_answer"
    }
  },
  "required": ["action"]
}

2. Auto-Generated from Tools:

If you don't specify a schema, one is generated from your tools automatically.

Schema Best Practices

Keep It Simple

Start minimal, add fields as needed:

// ✅ Good - Simple and clear
{
  "sentiment": "positive",
  "score": 0.9
}

// ❌ Overly complex for simple use case
{
  "analysis": {
    "sentiment": {
      "primary": "positive",
      "secondary": ["happy", "excited"],
      "scores": {
        "positive": 0.9,
        "negative": 0.05,
        "neutral": 0.05
      }
    }
  }
}

Use Enums for Fixed Values

Restrict to valid options:

{
  "priority": {
    "type": "string",
    "enum": ["low", "medium", "high", "critical"]
  }
}

Add Descriptions

Help the model understand fields:

{
  "temperature": {
    "type": "number",
    "description": "Temperature in Fahrenheit",
    "minimum": -100,
    "maximum": 150
  }
}

Mark Required Fields

Ensure critical fields are always present:

{
  "required": ["sentiment", "confidence"]
}

Validate Ranges

Constrain numbers to valid ranges:

{
  "rating": {
    "type": "integer",
    "minimum": 1,
    "maximum": 5
  }
}

Handle Arrays Properly

Define array item structure:

{
  "items": {
    "type": "array",
    "items": {
      "type": "object",
      "properties": {
        "name": {"type": "string"},
        "quantity": {"type": "integer", "minimum": 1}
      }
    }
  }
}

Common Use Cases

Form Extraction

Extract structured data from text:

{
  "type": "object",
  "properties": {
    "name": {"type": "string"},
    "email": {"type": "string", "format": "email"},
    "phone": {"type": "string"},
    "address": {
      "type": "object",
      "properties": {
        "street": {"type": "string"},
        "city": {"type": "string"},
        "zip": {"type": "string"}
      }
    }
  }
}

Invoice Processing

{
  "type": "object",
  "properties": {
    "invoice_number": {"type": "string"},
    "date": {"type": "string", "format": "date"},
    "total": {"type": "number"},
    "items": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "description": {"type": "string"},
          "quantity": {"type": "integer"},
          "price": {"type": "number"}
        }
      }
    }
  }
}

Customer Support Classification

{
  "type": "object",
  "properties": {
    "ticket_type": {
      "type": "string",
      "enum": ["bug", "feature_request", "question", "complaint"]
    },
    "urgency": {
      "type": "string",
      "enum": ["low", "medium", "high"]
    },
    "department": {
      "type": "string",
      "enum": ["technical", "billing", "sales", "general"]
    },
    "summary": {"type": "string"}
  }
}

Visibility

Schemas can be:

Private - Only you can use them
Public - Shared with community (coming soon)

Versioning

When updating schemas:

Create new schema with version suffix (v2)
Test thoroughly
Update adapters to use new schema
Keep old version for backward compatibility

Troubleshooting

Model Not Following Schema?

Check schema is well-defined
Ensure required fields are marked
Add descriptions to clarify field purposes
Test with simpler prompts

Schema Too Restrictive?

Make fewer fields required
Use looser type constraints
Add optional fields for edge cases

Next Steps

Use with Adapters - Link schemas to your models
Use with Agents - Structure agent outputs
API Reference - Schema API endpoints

Documentation