4 strategies to increase LLMs promote your LLMs on cost, latency and performance

of changing a significant number of jobs. Since the release of chatgpt in 2022, we have seen more and more products in the market using LLMS. However, there are still many improvements to be made in the way we use LLMS. Improving your speed with a fast LLM with LLM success and using cached tokens, for example, are two simple strategies you can use to greatly improve the performance of your LLM application.

In this article, I will discuss some specific strategies that you can incorporate into the way you create and plan your releases, which will reduce latency and cost, and also increase the quality of your responses. The goal is to introduce you to these specific methods, so that you can quickly apply them to your LLM application.

Why should you do well in your own haste

In most cases, you can have a short time working on a given LLM and it will produce sufficient results. However, in most groups, you never spend much time persevering quickly, which leaves a lot of energy on the table.

I argue that using some techniques that I will present in this article, you can easily improve the quality of your answers and reduce costs without much effort. Because a quick job with an LLM doesn't mean you're doing well, and in most cases, you can see a big improvement with a lot of effort.

Some breeding techniques

In this section, I'll cover some techniques you can use to improve your Refts.

Always save static content early

The first method I will always cover is to store static content early in your startup. By static content, I'm referring to content that stays the same when you make multiple API calls.

The reason you should store static content early is that all major LLM providers, such as anthropic, Google and Orandai, use tokens stored in the second. Cached tokens are tokens that have already been processed in a previous API request, and that can be processed cheaply and quickly. It varies from provider to provider, but saved input tokens tend to be priced around 10% of regular input tokens.

Cached tokens are tokens that have already been processed in a previous API request, and that can be processed cheaper and faster than regular tokens

That means, if you send the same match two times in a row, the second input tokens will cost only 1 / 10th of the first input tokens. This works because LLM providers cache the processing of these input tokens, which makes processing your new request cheaper and faster.

Basically, caching input tokens is done by storing a variable on the PRESPORT endpoint.

For example, if you have a long system fast with a query that varies from request to request, you should do something like this:

prompt = f"""
{long static system prompt}

{user prompt}
"""

For example:

prompt = f"""
You are a document expert ...
You should always reply in this format ...
If a user asks about ... you should answer ...

{user question}
"""

Here we have the static content first, before we put the dynamic content (user query) last.

In some cases, you want to feed the content of the document. If you are processing many different documents, you should store the contents of the document in the temporary end:

# if processing different documents
prompt = f"""
{static system prompt}
{variable prompt instruction 1}
{document content}
{variable prompt instruction 2}
{user question}
"""

However, suppose you process the same documents over and over again. In that case, you can make sure that the tokens for this document are also saved by making sure that no variables are included in that prompt:

# if processing the same documents multiple times
prompt = f"""
{static system prompt}
{document content} # keep this before any variable instructions
{variable prompt instruction 1}
{variable prompt instruction 2}
{user question}
"""

Note that cached tokens are normally activated only if the first 1024 tokens are the same for two requests. For example, if your static program start with the example above is shorter than 1024 tokens, you will not use cached tokens.

# do NOT do this
prompt = f"""
{variable content} < --- this removes all usage of cached tokens
{static system prompt}
{document content}
{variable prompt instruction 1}
{variable prompt instruction 2}
{user question}
"""

Your proposal should always be composed of a lot of content (a lot of content that varies from request to request), a lot of content (a lot of content that varies from request to request)

1. If you have a long system and user acceleration without variable, you should keep that first, and add the variable at the end of PRESSOPT
2. If you're managing text from documents, for example, and you're processing the same document twice, you should

It can be the content of the document, or if you have a long fast -> use the saver

One last question

Another technique you should use to improve LLM performance is to always include a user question at the end of your Prompt. Ideally, you edit it so you have your system prompt that contains all the standard commands, and a user prompt that only contains the user's question, as below:

system_prompt = ""

user_prompt = f"{user_question}"

In anthropic's fast developer scripts, a scenario that quickly enters the user at the end can improve performance by up to 30%, especially when using long scenarios. Including a question at the end makes it easier to model what you are trying to achieve, and will, in most cases, lead to better results.

You use a fast optimizer

Many times, when people write motivations, they become boring, inconsistent, contain unnecessary content, and lack structure. Therefore, you should always feed your speed with a speed optimizer.

The easiest activator you can use to renew LLM to Upgrade this fast {fast}, And it will quickly give you more organized, less unnecessary content, and more.

The best way, however, is to use some quick optimizer, like the one you can get from Opelai or anthropic gain. These optimizers are specifically requested by LLMS and are designed to optimize your output, and tend to produce better results. In addition, you should be sure to include:

• Details about the task you are trying to accomplish
• Examples of quick operations are successful, as well as import and export
• An example of tasks that quickly failed, with installation and removal

Providing this additional information will often lead to better results, and you will end up with better acceleration. In most cases, you'll spend about 10-15 minutes and end up with a quick fix. This makes using a fast optimizer one of the most low-cost approaches to improving LLM performance.

Benchmark llms

The LLM you apply for will have a significant impact on the performance of your LLM program. Different LLMS are good for different careers, so you need to try different LLMs for your specific application area. I recommend at least setting up access to major LLM providers like google Gemini, Openai, and Anthropic. Setting this up is really easy, and changing your LLM provider takes minutes once you already have the details set up. In addition, you can consider exploring open LLMSs as well, although they often require more effort.

Now you need to set a specific benchmark for the Career you are trying to achieve, and see which LLM works best. In addition, you should check the performance of the model regularly, because the major LLM providers periodically update their models, unless they come out with a new version. You should, too, be prepared to try any new models that come out of the major LLM providers.

Lasting

In this article, I've covered four different strategies you can use to improve the performance of your LLM program. I've discussed using cached tokens, having a query at the end of the Prompt, using fast optimizers, and building some LLM benchmarks. All of this is very easy to set up and do, and can lead to high performance increases. I believe that many similar and simple techniques exist, and you should always try to look at them. These topics are often explained in separate blog posts, where anthropic is one of the blogs that have helped me improve LLM performance the most.

👉 Find me in the community:

📩 Subscribe to my newsletter

🧑💻 Get in touch

🔗 lickEdin

🐦 X / Twitter

✍️ Medium

You can also read some of my articles: