I Need a Classifier but I Don't Have Training Data or an ML Team

TL;DR

You do not need an existing labelled dataset to evaluate a classifier. An LLM can generate examples offline, then Sparkient trains and deploys a lightweight model. Four public synthetic domains measure 0.886–0.951 macro F1 and 33–42ms batch-average time per item; the runner did not measure per-request p95. Production use still requires a clear policy, representative evaluation, and monitoring.

The Problem: The ML Cold Start

You need a classifier. Maybe it's content moderation for your community platform. Maybe it's ticket routing for your support queue. Maybe it's lead scoring for your sales pipeline.

The traditional path looks like this:

1. Collect data — months of historical examples, manually labelled by domain experts
2. Allocate ML capability — internal specialists, a partner, or founder time
3. Build infrastructure — training pipelines, model serving, monitoring
4. Iterate — feature engineering, hyperparameter tuning, evaluation
5. Deploy and maintain — model versioning, retraining schedules, drift detection

The timeline and cost depend on the existing data, team, evaluation burden, model, and deployment. The practical question is whether the decision deserves a custom ML project at all.

The Four Options

There are four practical approaches to getting a classifier running without existing training data or ML expertise. Each has real tradeoffs.

Option 1: Zero-Shot LLM Classification

The simplest approach. Send each input to an LLM with a classification prompt:

import openai

response = openai.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{
        "role": "user",
        "content": f"""Classify this support ticket into one of: billing, technical, account, general.

Ticket: {ticket_text}

Respond with only the category name."""
    }]
)
category = response.choices[0].message.content.strip()

Pros: Works immediately, no training data needed, handles nuance well. Cons: Current provider pricing and limits; model-, prompt-, and load-dependent latency; outputs can change across model versions. Best for: Prototyping or production where the measured quality, runtime, and operating cost fit.

Option 2: Few-Shot Prompting

Improve accuracy by including examples in the prompt:

prompt = """Classify this support ticket. Here are examples:

"My card was charged twice" → billing
"App crashes when I upload photos" → technical
"I need to change my email address" → account
"What's your return policy?" → general

Now classify: {ticket_text}

Respond with only the category name."""

Pros: Can improve project-specific quality when the examples are representative; easy to iterate by editing examples. Cons: Adds input tokens and can change cost and latency; the prompt grows with the number of examples. Best for: When a held-out comparison supports adding a small set of representative examples.

Option 3: Fine-Tuning a Foundation Model

Fine-tune a smaller model on your specific classification task:

# OpenAI fine-tuning
training_data = [
    {"messages": [{"role": "user", "content": ticket}, {"role": "assistant", "content": label}]}
    for ticket, label in labelled_examples
]

openai.fine_tuning.jobs.create(
    training_file=upload_file(training_data),
    model="gpt-4o-mini-2024-07-18"
)

Pros: Lower per-call cost than base models, potentially better accuracy on your specific domain. Cons: You need a task-specific dataset, pay training and inference or hosting costs, and take on provider or model operations. Data needs and latency are model-dependent. Best for: When you have labelled data and want better accuracy than prompting, but can tolerate per-call costs.

Option 4: LLM-as-Teacher Compilation

Use the LLM to generate labelled training data, then compile a standalone classifier:

1. Define decision options → "billing", "technical", "account", "general"
2. LLM generates hundreds of synthetic training examples
3. System trains a lightweight classifier (compiled model)
4. Model exports to ONNX, deploys to production
5. Production decisions: compiled normal path, with optional cloud escalation

Pros: Can start without an existing labelled dataset, provides a managed training workflow, targets a sub-100ms compiled path, avoids a live LLM call on the normal runtime path, and returns structured outputs. You still own evaluation, policy review, and production monitoring. Cons: Less flexible than a prompt, requires fixed options and retraining for learned-policy changes, and quality is task-specific. Best for: Repeated classifications with fixed options and a measured quality, latency, cost, privacy, reliability, or offline constraint.

Comparing the Options

| | Zero-Shot LLM | Few-Shot LLM | Fine-Tuning | LLM-as-Teacher | |--|:---:|:---:|:---:|:---:| | Training data needed | None | A small representative set | Provider- and task-dependent | Can begin with generated examples | | ML expertise needed | None | None | Some | None | | Setup time | Project-dependent | Project-dependent | Project-dependent | Train and evaluate before integration | | Published timing | Measure per-call latency | Measure per-call latency | Model and hosting dependent | 33–42ms batch-average time per item in public synthetic runs; measure p95 per workload | | Cost model | Token usage | Token usage | Provider or hosting usage | Plan credits | | Runtime LLM dependency | Every call | Every call | Depends on serving model | Normal path no; optional cloud escalation | | Output contract | Prompt constrained | Prompt constrained | Schema constrained | Structured decision response |

The Sparkient Workflow: Zero to Candidate Classifier

Here's the actual workflow for building a classifier without data or ML expertise using Sparkient:

Step 1: Describe Your Decision

You don't write code for this part — you describe what the decision means in plain English:

import httpx

API = "https://api.sparkient.ai/api/v1"
HEADERS = {"Authorization": "Bearer YOUR_API_KEY"}

response = httpx.post(f"{API}/decision-types", headers=HEADERS, json={
    "name": "content_moderation",
    "description": (
        "Decide whether user-generated content should be approved, sent for "
        "human review, or rejected. Approve content that is on-topic, constructive, "
        "and follows community guidelines. Send for review if the content is "
        "borderline — potentially offensive but context-dependent. Reject content "
        "that contains hate speech, explicit threats, spam, or illegal content."
    ),
    "options": ["approve", "review", "reject"],
    "reason_codes": [
        "on_topic", "constructive", "borderline_language", "needs_context",
        "hate_speech", "threats", "spam", "illegal_content"
    ]
})
decision_type_id = response.json()["id"]

The description states the decision criteria that the LLM teacher uses when proposing examples. Review the resulting labels against the policy.

Step 2: Generate Training Data (Automatic)

Sparkient's LLM teacher can propose candidate examples. The endpoint accepts at most 50 examples per request, and training requires at least 38 labelled examples for every option:

examples = []
for _ in range(6):
    response = httpx.post(
        f"{API}/decision-types/{decision_type_id}/examples/generate",
        headers=HEADERS,
        json={"count": 50},
    )
    response.raise_for_status()
    examples.extend(response.json())

readiness = httpx.get(
    f"{API}/decision-types/{decision_type_id}/training-readiness",
    headers=HEADERS,
).json()
if not readiness["ready"]:
    raise RuntimeError(readiness["issues"])

print(readiness["per_option"])

Inspect the generated examples across all options, especially ambiguous and consequential cases. Readiness checks quantity and class coverage; it does not establish label quality.

Step 3: Train the Model (Automatic)

One API call triggers the full training pipeline:

response = httpx.post(
    f"{API}/decision-types/{decision_type_id}/train",
    headers=HEADERS,
    json={"auto_deploy": True}
)
print(f"Training policy: {response.json()['policy_id']}")

Behind the scenes:

• Text encoder creates semantic embeddings from text features
• Classifier trains on the combined feature set
• Hyperparameters are tuned automatically
• The model exports to ONNX format
• If auto_deploy is set, the completed policy deploys automatically when any configured quality gate is met

Step 4: Use It (Under-100ms Compiled-Stage Target)

response = httpx.post(f"{API}/decide", headers=HEADERS, json={
    "decision_type": "content_moderation",
    "input": {
        "text": "Has anyone else noticed the new update breaks dark mode?",
        "user_id": "user_456"
    }
})

result = response.json()
# The response includes the decision, confidence, measured latency, and stage.

You now have a deployed candidate classifier trained from generated examples. It becomes production-ready only after it passes the project's representative evaluation and operational checks.

Adding Your Own Data Later

The LLM-as-teacher approach doesn't lock you out of using real data. As your product runs and you collect real examples:

1. Export decision logs — every decision Sparkient makes is logged with the input and output
2. Add real examples — upload actual classified examples from your production data
3. Retrain and compare — train on the reviewed mix, then check whether the fixed evaluation set actually improves

This is a natural progression: start with synthetic data when you have nothing, then enrich with real data as it becomes available.

When You Actually Need an ML Team

Be realistic about when the LLM-as-teacher approach isn't enough:

• Custom model architectures — if your problem requires a specialized neural network (image classification, time-series prediction, recommendation systems), you need ML expertise.
• High-consequence decisions — medical, safety-critical, legal, or other consequential uses require domain experts, extensive validation, and controls beyond a generic classifier workflow.
• Non-classification problems — if you need generation, ranking, or regression, a classifier won't help. Compilation works specifically for structured decisions with fixed output options.

Required quality depends on the class and consequence. Sparkient's public 0.886–0.951 macro-F1 range justifies evaluation in several domains, but it is not automatically sufficient for a production decision.

FAQ

Q: How accurate is a classifier trained on synthetic data vs real data? Across four validated public domains, Sparkient's compiled models achieve 0.886–0.951 macro F1 using synthetic noisy-data benchmarks. Content moderation achieves 0.900 macro F1 and 91.5% accuracy. The results do not prove equivalence to a human-labelled model; optional escalation must be evaluated separately.

Q: Can I define the decision options myself, or does the system choose them? You define them. The options, description, reason codes, and any hard rules are all specified by you. The system generates training data and trains a model to match your definition — it doesn't decide what the categories should be.

Q: What if my first model isn't accurate enough? You can add specific examples, refine the decision definition, add CEL rules for deterministic outcomes, or upload reviewed production examples. Each retraining run costs 2,000 credits; duration varies with data and configuration, and every new model should be checked on a fixed evaluation set.

Q: Do I need to know Python to use Sparkient? No. The API is a standard REST API — any language that can make HTTP requests works. There's also an MCP server for Claude, Cursor, and VS Code that lets you create decision types, generate examples, and train models from your IDE without writing any API integration code.

You do not need an existing labelled dataset to test the approach. Start with the free tier, train one candidate, and decide from measured quality, latency, credits, and effort.