Open models need a working system.

Our engineering perspective. Examples are illustrative, not client results or claims of measured model performance. Primary resources are linked in the text.

A model release can improve reasoning, speed or deployment flexibility overnight. Your business process does not automatically improve with it. Someone still has to decide where the model fits, what it may access, how its output is checked and what happens when execution stops halfway through.

Open models make that design space more interesting. They can give an operator more control over hosting, versioning and adaptation. They also bring obligations that a hosted API may otherwise absorb. The useful question is which arrangement serves the workflow, including its operational burden.

Two 2026 releases worth evaluating

Google’s Gemma 4 announcement introduced E2B, E4B, 26B mixture-of-experts and 31B dense variants under Apache 2.0. The range creates different deployment candidates, rather than a single universal choice. Exact hardware fit still depends on precision, context, concurrency and the serving implementation.

Mistral Small 4, announced on March 16, 2026, combines text and image inputs with configurable reasoning effort under Apache 2.0. Mistral documents 119 billion total parameters and 6 billion active parameters per token, or 8 billion including embedding and output layers. Active parameters are not a complete memory-sizing rule.

These are selected releases, reviewed for this article on September 10, 2026. This is not an exhaustive release feed or a Looqus benchmark. Publisher capability statements help build a shortlist. They do not establish that a model will interpret your catalog, respect your policy or outperform another model on your workflow.

Separate the layers before choosing the tools

The model generates a prediction. The serving layer loads it, accepts requests and manages execution resources. The harness controls the task around it: context assembly, tool calls, saved state, limits, approvals and recovery. The business application defines what the work means.

Those layers should have explicit interfaces. Replacing a serving backend should not change who may approve a refund. Switching a model should not erase the draft version a person reviewed. A connector’s successful HTTP response should not be the only evidence that an order reached the intended state.

vLLM’s structured-output documentation is a useful resource for constrained output formats. A schema can make output easier to parse. It cannot prove that a product identifier is real or that a proposed action is allowed. Validate those properties in application code against the authoritative records.

Give each task an execution contract

Start with a task record, not an unbounded instruction. It should identify the source event, the business object, a context version, the permitted tools and a completion check. Restrict each step to the information it needs. A product-feedback classifier has no reason to receive payment credentials or a customer’s entire order history.

# Illustrative control flow, not a runnable SDK example.
run = load_or_create(event_id)
context = read_authorized_sources(run.scope)
proposal = model.generate(context, output_schema)
validate_schema_and_business_rules(proposal)
save_versioned_proposal(run, proposal)
if requires_review(proposal):
    suspend_with_named_owner(run)
else:
    execute_once(run.action_key, proposal)
    verify_source_system_state(run)
record_outcome_or_exception(run)

The action key deserves particular attention. A worker may finish a request and lose the response. Blindly retrying can send a second message or create a second order. Use the downstream system’s idempotency mechanism where available. Otherwise reconcile the current state before retrying a consequential action.

Keep workflow state outside the conversation

A chat log is valuable evidence, but it is a poor substitute for an explicit state machine. “Awaiting review,” “approved version two” and “execution outcome unknown” should be distinct states. Store the approval actor, scope and artifact version. When an artifact changes, decide whether the approval remains valid rather than assuming it transfers.

LangGraph provides primitives for persistent, long-running orchestration and human intervention. Such tools can help implement the design. Choosing a framework does not choose your business rules, database isolation or operational response process.

Open weights do not mean free operation

Evaluate the full run cost: inference capacity, idle time, storage, monitoring, failed runs and human correction. A small high-volume workload may justify dedicated serving. A sporadic workload may be better served by an API. Quantization and batching can alter both quality and latency, so evaluate the deployed configuration rather than the model name alone.

Looqus’ commercial position is to cover agreed model and workflow infrastructure costs within the deployment and incremental-revenue arrangement. That makes efficiency our operating responsibility. It does not make compute disappear.

The practical advantage of an open model is optionality. The practical advantage of a well-designed harness is that this optionality can be exercised without losing the business process around it.