AI Decision Models: Distinct from LLMs

Decision models have recently emerged as a distinct paradigm in artificial intelligence. One prominent example is Jev by TypeSafe AI. Unlike generative models, Jev outputs structured options, scores, and confidence metrics rather than raw text.

While conventional AI models have long handled predictions and classifications, Jev is engineered to make structured decisions that software systems can integrate directly. This approach provides an alternative pathway for developing AI applications, particularly when a system must assess information and determine the subsequent action.

How does Jev work?

In a standard LLM workflow, you submit a prompt and receive generated text. Even when requesting JSON, the application must parse and validate the response before utilization.

Jev operates differently. You supply context, define the specific questions to be answered, and specify the expected output format. Jev then returns structured results that your application can process immediately.

Jev supports three primary question types:

• Choice: Selects an option from a predefined list, such as billing, technical support, or sales.
• Score: Rates an item against an ordered scale, such as low, medium, or high priority.
• Yes or no: Estimates the probability of a statement's truth, such as whether a support ticket requires human intervention.

Multiple questions regarding the same input can be posed in a single request. For instance, a support application could classify a ticket, assess its urgency, and determine if human review is required simultaneously. Jev delivers these results in a structured format, enabling the application to determine the next course of action.

Decision models vs. LLMs

LLMs and decision models serve distinct purposes. An LLM is capable of explaining decisions, generating content, writing code, or reasoning through complex tasks. Jev is designed for focused decision-making with defined outputs.

The distinction does not imply that LLMs cannot make decisions; they can certainly classify information and return structured data. However, Jev is purpose-built for this task, eliminating the need for applications to interpret free-form responses.

This makes it particularly useful for repeated, well-defined decisions. For tasks demanding detailed explanations, creative output, or complex multi-step reasoning, an LLM remains the more suitable tool.

What are confidence scores?

A key feature of Jev is that its outputs include probabilities and confidence metrics. These values assist developers in assessing the model's certainty in a given answer.

For example, a customer support application might query whether a ticket should be escalated. If the model indicates a high probability that escalation is necessary, the application could route the ticket to a senior support team. Conversely, if the result is uncertain, it might flag the ticket for human review.

However, confidence does not guarantee accuracy. A model may be highly confident yet still make an incorrect decision. Developers should validate results against real-world examples and set thresholds appropriate to the consequences of each decision.

It is also crucial to distinguish the probability of an answer from a separate confidence score. The specific meaning depends on the output type. Applications should adhere to the model's documentation rather than assuming all returned numerical values represent the same metric.

Where can decision models be useful?

Classification and routing

Many applications require the sorting of incoming data before processing. A support platform, for instance, might categorize requests by topic and route them to the appropriate team. Similarly, business applications can classify documents, emails, or customer feedback.

Jev can return a category from a predefined list, enabling the application to route items without needing to parse a generated paragraph.

Scoring and evaluation

Decision models can also evaluate information against a defined scale. For example, an application could rate a support ticket by urgency, score a document for relevance, or assess whether a response meets specific criteria.

These results can be used to rank items, prioritize work, or identify cases requiring further review. The quality of these decisions depends on the model and how precisely the criteria define the task.

AI agents and tool selection

AI agents utilize models to determine actions. An agent might select a tool, classify a request, or determine if a task requires additional information.

While a general-purpose LLM can make these decisions as part of its reasoning process, a decision model can handle narrower choices before the agent executes an action.

For example, a custom workflow using Hermes Agent could leverage Jev to classify an incoming task before passing it to the agent. It could also assist in determining which workflow should handle a request. This would require a custom integration rather than relying on assumed native support.

Decision models should not be relied upon solely as security controls. If an agent can execute commands or modify files, permissions and deterministic rules must enforce allowed actions. While a model's classification can inform this process, it should not replace it.

Alternatives to Jev

Jev is part of a broader category of models designed to produce structured decisions. OpenJev and Laya are two other projects worth exploring when comparing approaches.

OpenJev

OpenJev is an open project related to the Jev approach. It is worth examining if you wish to explore an implementation outside of a hosted service. Before utilization, verify its current availability, hardware requirements, supported features, and license terms.

Laya

Laya is an open-source decision model from Convai Innovations. It focuses on tasks such as selecting from predefined options, rating information, and answering yes-or-no questions. It may be relevant if you intend to evaluate a model that can run in your own environment.

These projects should be compared based on actual performance, not just stated capabilities. Use consistent test data and measure accuracy, latency, resource requirements, and the alignment of confidence estimates with real-world outcomes.

Can you run decision models locally?

Running a model locally offers greater control over the software environment and data processing location. However, deployment options vary by project. Jev is offered via an API, while open-source alternatives may provide options for running inference on your own hardware.

Hardware requirements depend on model size, memory usage, and throughput. Some models can operate on a CPU, while others benefit from a GPU, especially when processing high volumes of requests.

If you wish to experiment with local AI models, DaDesktop provides cloud desktops with dedicated GPU options. You can utilize these for development and testing without purchasing a high-end GPU for your local computer.

Learn more about running local LLMs or explore AI agent environments.

When should you use a decision model?

Decision models are beneficial when an application requires a defined answer that can trigger a specific next step. They complement LLMs by handling classification, scoring, and routing, while generative models handle language and complex reasoning.

The optimal choice depends on your application. Test the model against your own data, compare it with conventional classifiers and LLMs, and measure accuracy, latency, and cost. For critical decisions, include human review and safeguards where necessary.

Want to discuss your use case? Contact us and we'll guide you in the right direction.