Run the Mythos Advanced Code Model locally with llama.cpp and Pi

# Introduction

Qwythos-9B-Claude-Mythos-5-1M is a 9B logic and code model based on Qwen3.5, designed for local coding workflows, agent development, and remote content operations. What makes it interesting is that it's small enough to run on consumer hardware, while still being powerful enough to help with practical coding tasks.

In this guide, I will show you how to use Qwythos advanced model of Mythos in the environment using llama.cppand connect it to Pi so you can use it as a local coding agent. I'll be using an RTX 4070 Ti Super with 16GB of VRAM, which allows me to run the Q6_K MTP benchmark comfortably. If you have an 8GB GPU, I recommend starting with the Q4_K_M variant because it offers a better balance between quality, speed, and memory usage.

# Includes llama.cpp

First, install the llama.cpp command line interface (CLI). This will give you access to llama command, including llama servewhich we will use to run the model locally.

curl -LsSf  | sh

Next, add an install directory to your shell path so that your terminal can find the llama command:

echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.bashrc
source ~/.bashrc

To verify that the installation worked, use:

If everything is installed correctly, you should see the available llama.cpp commands and options.

# Setting up the Hug Face Cache

set the A Hugging Face cache somewhere with enough free storage. This is especially useful on cloud machines where the default home directory has limited disk space.

export HF_HOME="/workspace/huggingface"
mkdir -p "$HF_HOME"

To continue with it in new terminal sessions, add it to your shell configuration:

echo 'export HF_HOME="/workspace/huggingface"' >> ~/.bashrc
source ~/.bashrc

# Starting Qwythos MTP Model

Run the Q6_K MTP GGUF model with all available GPU layers:

llama serve 
  -hf "empero-ai/Qwythos-9B-Claude-Mythos-5-1M-GGUF:MTP-Q6_K" 
  --alias "qwythos-9b-mtp" 
  --host 0.0.0.0 
  --port 8910 
  --n-gpu-layers all 
  --ctx-size 100000 
  --parallel 1 
  --batch-size 1024 
  --ubatch-size 512 
  --flash-attn on 
  --cache-type-k q8_0 
  --cache-type-v q8_0 
  --spec-type draft-mtp 
  --spec-draft-n-max 6 
  --threads 12 
  --threads-batch 24 
  --temp 0.6 
  --top-p 0.95 
  --top-k 20 
  --repeat-penalty 1.05 
  --jinja 
  --perf

When you first run this command, llama.cpp will download the model files from Hugging Face.

After that, it will load the model into the GPU memory and start the local server.

If the model is working, open the local interface in your browser:

This also gives you an OpenAI compatible endpoint for the local API in:

The Q6_K variant offers better output quality, but also uses more memory. If you experience VRAM or RAM issues, lower it --ctx-size first. If that's not enough, try the Q4_K_M variant instead.

Here are the most important flags in the command:

On my RTX 4070 Ti Super, I was getting about 81.74 tokens per second. In my testing, the model was also able to think of a job, call the necessary tools, and retrieve the latest gold price when used within a local agent's workflow.

# Includes Pi and llama.cpp Integration

The Pi can connect to the local llama.cpp server and use the model as a coding agent. I pi-llama The plugin is designed to connect the Pi to a local llama.cpp server running, without requiring an external API key.

Install the Pi:

curl -fsSL  | sh

Install the llama.cpp plugin for the Pi:

pi install git:github.com/huggingface/pi-llama

Create a small test project:

mkdir new-project
cd new-project

By default, the plugin looks for llama.cpp on port 8080. For this guide, our local server is running on port 8910, so we need to set the correct API address before starting the Pi:

export LLAMA_BASE_URL="

Now turn on the Pi:

Inside the Pi, type:

Choose a local model alias:

You should see the model listed inside the Pi, and once it's selected, you can start using it as a local coding agent.

# Testing the Advanced Code Model of Legends with Pi

Now it's time to test the local model in actual coding tasks inside the Pi. I used two simple but effective functions: a browser game and a small Python CLI tool.

// Building a Simple Browser Game

Start with a command that asks the Pi to run a small browser game called “Beat the AI.”

Create a simple browser game called “Beat the AI”.

The player has 30 seconds to answer short pattern recognition questions. Each correct answer increases the score by one. Display a countdown timer, score display, progress bar, and final results screen.

Requirements:
– Use only HTML, CSS, and vanilla JavaScript.
– Generate at least three types of questions, such as number patterns, quick math, and word order.
– Add a restart button after the game ends.
– Make the user interface feel more playful and polished.
– Keep all code easy to understand and avoid unnecessary files.

Test the game in a browser before finishing.

In my testing, Pi created a full game in a single HTML file with embedded CSS and JavaScript, keeping the project simple and easy to test.

Pi then summarized what he had built, including a 30-second countdown, score tracking, a progress bar, question types, and a restart button.

After that, I opened the game in the browser to make sure it works properly.

The result was a polished mini-game that:

• Countdown timer
• Score display
• Progress bar
• Multiple question types
• The flow restarts after the end of the game

This is a great example of how a model can handle a complete endpoint function locally without requiring an external API.

// Creating a CSV-to-Excel Python CLI

In the second experiment, ask Pi to create a small Python CLI that converts a CSV file to Excel .xlsx file.

Build a simple Python CLI that converts a CSV file to an Excel .xlsx file, accepts input and output file paths as arguments, saves column headers, validates missing files, and prints clear success or error messages. Before finishing, create a small dummy CSV file with sample data, use it to test the CLI, confirm that the Excel file is created correctly, and summarize the test result.

Pi created a Python script named after it csv2excel.pycreated a sample CSV file, ran the test command, and summarized the results.

python csv2excel.py sample_data.csv output.xlsx

In my run, the summary showed that the script:

• Accepted input and output file paths.
• Saved column headings.
• Well converted sample data.
• Handle missing files with clear error messages.
• Handles poor output methods.

I also opened the generated Excel file to confirm that the output was correct.

The sample output preserved the lines and headers correctly, which ensured that the CLI worked as expected.

# Final thoughts

I really like this little local coding model. It's fast, accurate enough for everyday coding tasks, and doesn't require a large amount of VRAM to be useful. You can use it with Pi, Claude Code, OpenCodeand any code setup that supports OpenAI-compatible or Anthropic-compatible endpoints.

For basic front-end applications, Python scripts, CLI tools, and quick prototypes, it works surprisingly well. You can build useful projects locally without relying on external APIs or paying for the entire application.

For best results, I highly recommend adding web search capabilities, Context7and other useful Pi integrations. You can also check out my full guide to optimizing the Pi coding agent setup here: How to Setup Kimi K2.7 Code with Pi: The Ultimate AI Coding Environment.

Abid Ali Awan (@1abidiawan) is a data science expert with a passion for building machine learning models. Currently, he specializes in content creation and technical blogging on machine learning and data science technologies. Abid holds a Master's degree in technology management and a bachelor's degree in telecommunication engineering. His idea is to create an AI product using a graph neural network for students with mental illness.