LoRA Adapters

Deploy and use fine-tuned LoRA adapters on MLStack. Adapters let you customize base models for your specific use case without hosting infrastructure.

What are LoRA Adapters?

LoRA (Low-Rank Adaptation) is an efficient fine-tuning technique that adds small trainable parameters to a frozen base model. Instead of fine-tuning billions of parameters, LoRA only trains ~1% of them.

Benefits

Efficient - Train in minutes, not hours
Small - Adapter files are only 10-100MB
Stackable - Switch between adapters instantly
Cost-effective - No need to deploy full custom models

Use Cases

Domain Adaptation - Medical, legal, financial terminology
Style Transfer - Write in specific tones or formats
Task Specialization - Q&A, summarization, code generation
Brand Voice - Match your company's communication style

Supported Base Models

Qwen/Qwen2.5-14B-Instruct
meta-llama/Llama-3.1-8B-Instruct
meta-llama/Llama-3.2-1B-Instruct
meta-llama/Llama-3.2-3B-Instruct
mistralai/Mistral-7B-Instruct-v0.3

More models coming soon! Email [email protected] to request a model.

Training LoRA Adapters

Before uploading to MLStack, you need to train your adapter using Hugging Face PEFT (Parameter-Efficient Fine-Tuning).

Required Format

MLStack expects adapters in PEFT format with these files:

adapter_config.json - LoRA configuration
adapter_model.safetensors (or adapter_model.bin) - Trained weights

Training with PEFT

PEFT is the standard library for LoRA training.

from peft import LoraConfig, get_peft_model, TaskType
from transformers import AutoModelForCausalLM, AutoTokenizer, Trainer, TrainingArguments

# Load base model
model = AutoModelForCausalLM.from_pretrained(
    "meta-llama/Llama-3.1-8B-Instruct",
    load_in_8bit=True,  # Reduce memory usage
    device_map="auto"
)

tokenizer = AutoTokenizer.from_pretrained("meta-llama/Llama-3.1-8B-Instruct")

# Configure LoRA
lora_config = LoraConfig(
    r=16,                           # Rank
    lora_alpha=32,                  # Scaling factor
    target_modules=["q_proj", "v_proj", "k_proj", "o_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM
)

# Apply LoRA
model = get_peft_model(model, lora_config)
model.print_trainable_parameters()  # Shows only 0.1-1% params are trained

# Train
trainer = Trainer(
    model=model,
    train_dataset=train_dataset,
    args=TrainingArguments(
        output_dir="./my-adapter",
        num_train_epochs=3,
        per_device_train_batch_size=4,
        learning_rate=2e-4,
    )
)

trainer.train()

# Save adapter (creates adapter_config.json + adapter_model.safetensors)
model.save_pretrained("./my-adapter")

Alternative Training Tools

Other tools that output PEFT-compatible format:

Axolotl - Config-based training with presets
Unsloth - 2x faster training, lower memory
TRL - Reinforcement learning fine-tuning

All use PEFT under the hood and save in the same format.

Dataset Format

Format your training data as instruction-following examples:

JSONL format:

{"instruction": "Classify sentiment", "input": "This is great!", "output": "Positive"}
{"instruction": "Classify sentiment", "input": "This is terrible", "output": "Negative"}

Or use Hugging Face datasets:

from datasets import load_dataset
dataset = load_dataset("your-username/your-dataset")

Training Tips

Start small - 100-1000 examples is often enough
Quality > Quantity - Clean, diverse data beats large noisy datasets
Monitor loss - Stop when validation loss plateaus

Required Output Files

After training, you need these files:

my-adapter/
├── adapter_config.json       # LoRA configuration
└── adapter_model.safetensors # Adapter weights (or .bin)

Example adapter_config.json:

{
  "peft_type": "LORA",
  "r": 16,
  "lora_alpha": 32,
  "lora_dropout": 0.05,
  "target_modules": ["q_proj", "v_proj", "k_proj"],
  "base_model_name_or_path": "meta-llama/Llama-3.1-8B-Instruct"
}

Creating an Adapter

Once trained, register your adapter on MLStack.

Create via UI

Go to My Adapters
Click Create Adapter
Fill in Basic Information:
- Adapter Name - Unique identifier (lowercase, hyphens only, e.g., customer-support-v1)
- Base Model - Select the model you fine-tuned from
- Output Schema (optional) - Select if using structured output
Add Description (optional) - Explain what your adapter does
Add Use Cases (optional) - Specific scenarios where it excels
Add Limitations (optional) - Known constraints or biases
Add Tags (optional) - Categorize your adapter (max 10 tags)
Upload Adapter Files:
- Adapter Config - Upload adapter_config.json (required)
- Adapter Weights - Upload .safetensors or .bin file (required)
- Tokenizer Files (optional) - Only if you added custom tokens
Set Visibility - Private (only you) or Public (all users)
Click Create Adapter

Your adapter is ready to use immediately!

File Requirements

Required files:

adapter_config.json - LoRA configuration
adapter_model.safetensors (or .bin) - Trained weights

Optional tokenizer files (only if you added custom tokens):

tokenizer_config.json
tokenizer.json
special_tokens_map.json
tokenizer.model (for SentencePiece models)

Size limits:

Max 500MB per adapter
Supported formats: .safetensors (recommended), .bin

Running Inference

Use your adapters via API or playground.

Test in Playground

Quick testing:

Go to your adapter page
Scroll to Test Inference
Enter a prompt
Click Run Inference
Wait for the result

API Usage

import requests
import time

API_KEY = "mlsk_your_key_here"

# Submit job
response = requests.post(
    "https://mlstack.io/api/llm/predict",
    headers={
        "Authorization": f"Bearer {API_KEY}",
        "Content-Type": "application/json"
    },
    json={
        "prompt": "Classify: This product exceeded expectations!",
        "adapter_name": "sentiment-classifier-v2",
        "max_new_tokens": 100,
        "temperature": 0.7
    }
)

job_id = response.json()["job_id"]

# Poll for results
while True:
    result = requests.get(
        f"https://mlstack.io/api/llm/result/{job_id}",
        headers={"Authorization": f"Bearer {API_KEY}"}
    )

    data = result.json()
    if data["status"] == "completed":
        print(f"Output: {data['output']}")
        break

    time.sleep(1)

Structured Output

Guarantee JSON output by linking an output schema to your adapter.

What is Structured Output?

Force models to return JSON in your exact format:

Without schema:

The sentiment is positive with 95% confidence

With schema:

{
  "sentiment": "positive",
  "confidence": 0.95
}

Create and Link Schema

Go to Schemas → Create Schema
Define your JSON structure:

{
  "type": "object",
  "properties": {
    "sentiment": {
      "type": "string",
      "enum": ["positive", "negative", "neutral"]
    },
    "confidence": {
      "type": "number",
      "minimum": 0,
      "maximum": 1
    }
  },
  "required": ["sentiment", "confidence"]
}

Go to your adapter → Edit → Output Schema → Select your schema
Save changes

All inference now returns guaranteed JSON!

Use via API

response = requests.post(
    "https://mlstack.io/api/llm/predict",
    headers={"Authorization": f"Bearer {API_KEY}"},
    json={
        "adapter_name": "sentiment-analyzer",
        "prompt": "This is amazing!"
    }
)

# Response follows schema automatically
result = poll_result(job_id)
print(result["output"])
# {"sentiment": "positive", "confidence": 0.92}

Common Schema Examples

Entity Extraction:

{
  "type": "object",
  "properties": {
    "entities": {
      "type": "array",
      "items": {
        "type": "object",
        "properties": {
          "text": {"type": "string"},
          "type": {"type": "string", "enum": ["person", "company", "location"]}
        }
      }
    }
  }
}

Classification:

{
  "type": "object",
  "properties": {
    "category": {"type": "string"},
    "confidence": {"type": "number"},
    "reasoning": {"type": "string"}
  },
  "required": ["category", "confidence"]
}

Private Adapters

Default for all uploads. Only you can use them.

Public Adapters

Coming soon! The public adapters marketplace is under development.

Currently, if you mark an adapter as Public:

Others can use it if they know the adapter name (via API or direct link)
Your adapter won't appear in search or browse yet
Once the marketplace launches, your public adapter will automatically be visible to all users

To make an adapter public:

Go to your adapter → Edit
Toggle Public visibility
Add tags and detailed description for future marketplace visibility
Save changes

Best Practices

Naming

Use clear, descriptive names: medical-ner-v1, customer-support-llama3
Include version suffixes: v1, v2, 2024-03
Avoid generic names: my-adapter, test

Documentation

Explain what your adapter does
Provide example prompts
Note any special formatting requirements
List expected input/output format

Testing

Test with diverse inputs before deploying
Verify structured output schemas work correctly
Compare against base model to confirm improvement
Monitor usage and performance after deployment

Versioning

Create new adapters for major changes (v1 → v2)
Keep old versions available for compatibility
Document changes between versions

Next Steps

Create AI Agents - Use adapters in agentic workflows
Structured Output - Deep dive into schemas
API Reference - Complete endpoint docs
Pricing & Credits - Cost details and optimization

Documentation

LoRA Adapters

What are LoRA Adapters?

Benefits

Use Cases

Supported Base Models

Training LoRA Adapters

Required Format

Training with PEFT

Alternative Training Tools

Dataset Format

Training Tips

Required Output Files

Creating an Adapter

Create via UI

File Requirements

Running Inference

Test in Playground

API Usage

Structured Output

What is Structured Output?

Create and Link Schema

Use via API

Common Schema Examples

Visibility & Sharing

Private Adapters

Public Adapters

Best Practices

Naming

Documentation

Testing

Versioning

Next Steps