Generation of a Structured Structured Language Model

# Introduction

Usually, when you ask LLM – the abbreviation “A Large Language Model“- for convenience, structured output like JSON objects, for example, a combination of fast composition and a “pinch” of luck is needed. Otherwise, it may be difficult to get the model to get the smooth output you expect. Or so it was, until a novel open source library arrived on the scene: frame.

This library is designed to prevent common problems that LLMs face in these output-oriented use cases, such as hallucinations. More precisely, it introduces a certain degree of deterministic certainty in the process of producing the output.

Let's reveal what it is outlines allows us to do with this illustrated article, where we will show working examples in Python!

# Use Case 1: Multiple Choice Segmentation for Sentiment Analysis

Before diving completely into the first use case, you may be wondering. How do you do it outlines how does it work and how does it ensure correctness in the results of a structured model? At the logic level, it hides “systematically illegal” tokens during production instead of trying to fix bad text once it's done. This makes it almost impossible to break the rules under a particular desired output format.

Let's see the first example where we create an analysis pipeline for customer support tickets, and we want indeed one option from a limited, authoritative list of possible options. This is similar to the division problem, too generate.choice() function that helps us simulate, by forcing the nearest model to choose one of the predefined names or classes.

But first, let's put it next to the transformers to load previously qualified LLMs:

pip install outlines[transformers]

The following code is used outlines.from_transformers() loading a pre-trained model assisted by the Hugging Face auto classes into the model and its associated tokenizer. But the icing on the cake is: both are wrapped in outlines something that will later help tell the model what to find. In the decision stage, we not only pass the user request asking to split the update, but also a Literal an object that contains output constraints a model must restrict to:

import outlines
from transformers import AutoTokenizer, AutoModelForCausalLM
from typing import Literal

# 1. Loading the backend using standard Transformer-based models
model_name = "microsoft/Phi-3-mini-4k-instruct"

# We use outlines to load the model with its from_transformers() function
model = outlines.from_transformers(
    AutoModelForCausalLM.from_pretrained(model_name),
    AutoTokenizer.from_pretrained(model_name)
)

# 2. Calling the model directly, passing our approved strings as type constraints
sentiment = model(
    "Classify the sentiment of this customer review: 'I've been waiting two weeks for my delivery and it's still missing.'",
    Literal["Positive", "Negative", "Neutral"]
)

print(sentiment)

Output:

A word of caution here: although the reality we've described is part of Python's built-in typing module, rather than outlinesour out-of-the-box library still takes control of the model here: both the model and the tokenizer are wrapped in an object that enforces standard Python types, creating a limited state machine under the hood that limits the output to only the given options.

# Use Case 2: JSON Object Generation

This example first defines a Pydantic object that defines the desired structure of a JSON object that defines a fictional character with a name, description, and age. It then uses the prepackaged outlines model, passing a character object to validate the generated output follows this requested JSON object structure:

from pydantic import BaseModel

# 1. Define a Pydantic model for the desired JSON structure
class Character(BaseModel):
    name: str
    description: str
    age: int

# 2. Using the outlines-wrapped model to generate a JSON output conforming to the Pydantic model
json_output = model(
    "Generate a JSON object describing a fictional character named 'Anya'.",
    Character,
    max_new_tokens=200
)

print(json_output)

Output:

{ "name": "Anya", "description": "Anya is a young, adventurous woman with a passion for exploring new places and meeting new people. She has long, curly hair and bright green eyes that sparkle with curiosity. Anya is always eager to learn and loves to share her knowledge with others. She is kind-hearted and always willing to lend a helping hand to those in need. Anya's favorite hobbies include hiking, reading, and playing the guitar. She is a free spirit who values freedom and independence above all else." ,"age": 25 }

# Use Case 3: Pure JSON Generation for REST APIs

This third example, also related to JSON, is similar to the previous one but in a slightly different context. Imagine you're building an API that requires well-defined JSON payloads to update a database. Asking a standard LLM to get this output will often return annoying, trailing characters like commas that might crash a JSON attacker.

With frameworks, we redefine our JSON payload upload schema with a custom class object based on Pydantic.

from pydantic import BaseModel
from typing import Literal
import json

class ServerHealth(BaseModel):
    service_name: str
    uptime_seconds: int
    status: Literal["OK", "DEGRADED", "DOWN"]

# 1. Outlines should produce a raw string guaranteed to be valid JSON
raw_json_string = model(
    "Report the current status of the main Auth database.",
    ServerHealth,
    max_new_tokens=50
)

print(type(raw_json_string))  # This will just print: 

# 2. Pretty-printing
parsed_json = json.loads(raw_json_string)
print(json.dumps(parsed_json, indent=2))

Output:

{
  "service_name": "auth_db_status",
  "uptime_seconds": 1623456789,
  "status": "OK"
}

# Closing Remarks

Since LLMs are trained conversationalists who can't break syntax or manipulate to “sound human” in their conversations with us, getting them to produce reliable, structured output as pure JSON objects can feel like a pain. It presents a new, open source library that introduces deterministic assurance to the LLM output generation process for better, more reliable generation of structured outputs. This article has shown three simple but useful cases for beginners with this interesting tool.

Iván Palomares Carrascosa is a leader, author, speaker, and consultant in AI, machine learning, deep learning and LLMs. He trains and guides others in using AI in the real world.