Language Model Evaluation with GraphEval

# Introduction

Hallucinations are one of the most well-known problems that major language models (LLMs) he may have when making the answers. They occur when the model produces an answer that is false, irrational, or just made up, usually due to the model's lack of internal knowledge of the issue.

Although many solutions have emerged in recent years to address the problem of counterfactual models, frameworks for testing self-diagnostic methods have received relatively little study. One recent research by Amazon researchers propose to use knowledge graphs as a way to analyze and identify misconceptions that occur in LLMs. The framework presented in the study is named GraphEval.

In this article, we'll take a gentle, practical approach to demonstrating the building blocks of GraphEval by using a simulation, a lightweight code example that you can easily try on your machine.

# GraphEval Briefly

GraphEval proposes graphs of information to identify and demonstrate manipulation in the results produced by LLM. Unlike traditional performance metrics that provide a single score to evaluate factors such as accuracy, certainty, and so on, GraphEval uses a two-stage evaluation process that emphasizes interpretability, that is, providing information about where dreaming actually occurred.

To do this, GraphEval looks at two parameters:

• Creating an information graph from the generated model response. A graph contains three semantics of the form (Subject, Relationship, Object)where topics and objects correspond to nodes, and relationships correspond to edges connecting those nodes.
• Each of the three tests in the knowledge graph is constructed against the source context (ground truth knowledge body) by using a natural language model (NLI). Any triple that cannot be included in the context according to the NLI engine – because it is negative or neutral – is marked as detected.

# Illustrating GraphEval with a Code Example

Before starting the code that simulates the use of the GraphEval framework, let's make sure we have the necessary libraries installed:

!pip install -q transformers networkx matplotlib torch

The purpose of the code example we're going to walk through is to illustrate how the GraphEval method works, so we'll replace sections that would require a heavy computing load in a real-world scenario with some simulations, some that are lightweight.

Accordingly, we will simulate a ground truth knowledge base (context) that is assumed to contain factual information. In a production setting, this ground truth information can come from, for example, retrieving relevant documents from a vector database for an advanced retrieval generation (RAG) program. For simplicity, here we directly construct the context of the ground truth and store it in it source_context.

# The ground-truth context provided to the LLM
source_context = (
    "GraphEval is a hallucination evaluation framework based on representing information "
    "in Knowledge Graph (KG) structures. It acts as a pre-processing step and utilizes "
    "out-of-the-box NLI models to detect factual inconsistencies."
)

Now, suppose the following is an original LLM response to a user prompt such as “briefly explain what GraphEval is”. To begin the first phase of the evaluation process, we can ask the LLM assistant to create an information graph from that response. Both the response and the trace information used to obtain the information graph are shown below:

# The generated response we want to evaluate (contains a hallucination)
llm_output = (
    "GraphEval is an evaluation framework that uses Knowledge Graphs. "
    "It requires a highly expensive, enterprise-level server farm to operate."
)

# Prompt template that would theoretically be passed to a local/free LLM (e.g. Mistral-7B)
KG_EXTRACTION_PROMPT = f"""
You are an expert information extractor. Extract the core information from the following text as a Knowledge Graph.
Return the output strictly as a Python list of tuples in the format: (Subject, Relationship, Object).

Text: {llm_output}
"""

Again, for simplicity and to bypass the heavy computational load of running a large LLM locally, suppose the following three graphs are available:

# Simulated extraction to bypass the heavy computational load of running a massive LLM locally
extracted_triples = [
    ("GraphEval", "is", "evaluation framework"),
    ("GraphEval", "uses", "Knowledge Graphs"),
    ("GraphEval", "requires", "expensive enterprise server farm")
]

print("Extracted Triples:")
for t in extracted_triples:
    print

Extracted Triples:
('GraphEval', 'is', 'evaluation framework')
('GraphEval', 'uses', 'Knowledge Graphs')
('GraphEval', 'requires', 'expensive enterprise server farm')

from transformers import pipeline

# Loading the open-source NLI model
print("Loading DeBERTa NLI model...")
nli_evaluator = pipeline("text-classification", model="cross-encoder/nli-deberta-v3-small")

def evaluate_triple(context, triple):
    subject, relation, obj = triple
    hypothesis = f"{subject} {relation} {obj}"

    # Checking if the context entails the hypothesis
    result = nli_evaluator({"text": context, "text_pair": hypothesis})

    # NLI models normally output: 'entailment', 'neutral', or 'contradiction'
    label = result['label'].lower()

    # In GraphEval, anything other than 'entailment' is flagged as a hallucination
    is_hallucinated = label != 'entailment'

    return is_hallucinated, label, hypothesis

# Running the evaluation pipeline
evaluation_results = []

print("n--- GraphEval Results ---")
for t in extracted_triples:
    is_hallucinated, nli_label, hypothesis = evaluate_triple(source_context, t)
    evaluation_results.append((is_hallucinated, nli_label))

    status = "🚨 HALLUCINATION" if is_hallucinated else "✅ GROUNDED"
    print(f"{status} | Triple: {t} | NLI Output: {nli_label}")

--- GraphEval Results ---
✅ GROUNDED | Triple: ('GraphEval', 'is', 'evaluation framework') | NLI Output: entailment
✅ GROUNDED | Triple: ('GraphEval', 'uses', 'Knowledge Graphs') | NLI Output: entailment
🚨 HALLUCINATION | Triple: ('GraphEval', 'requires', 'expensive enterprise server farm') | NLI Output: neutral

import networkx as nx
import matplotlib.pyplot as plt
import matplotlib.patches as mpatches

def visualize_grapheval(triples, eval_results):
    G = nx.DiGraph()
    edge_colors = []

    for (triple, res) in zip(triples, eval_results):
        sub, rel, obj = triple
        is_hallucinated = res[0]

        G.add_node(sub)
        G.add_node(obj)
        G.add_edge(sub, obj, label=rel)

        # Color-code the edges based on the NLI evaluation
        edge_colors.append('red' if is_hallucinated else 'green')

    # Set up the plot
    plt.figure(figsize=(10, 6))
    pos = nx.spring_layout(G, seed=42)

    # Draw nodes
    nx.draw_networkx_nodes(G, pos, node_color="lightblue", node_size=2500)
    nx.draw_networkx_labels(G, pos, font_size=10, font_weight="bold")

    # Draw edges and labels
    nx.draw_networkx_edges(G, pos, edge_color=edge_colors, width=2.5, arrowsize=20)
    edge_labels = nx.get_edge_attributes(G, 'label')
    nx.draw_networkx_edge_labels(G, pos, edge_labels=edge_labels, font_color="black")

    # Add legend
    green_patch = mpatches.Patch(color="green", label="Grounded (Entailment)")
    red_patch = mpatches.Patch(color="red", label="Hallucination (Neutral/Contradiction)")
    plt.legend(handles=[green_patch, red_patch], loc="lower right")

    plt.title("GraphEval Hallucination Map", fontsize=14, fontweight="bold")
    plt.axis('off')
    plt.tight_layout()
    plt.show()

# Render the knowledge graph
visualize_grapheval(extracted_triples, evaluation_results)