The Guide for Following Telephone Use in the LLM applications

Photo by writer | Ideogram.ai

Obvious Introduction

When major applications for model language is formed, the money tokens are made. If you ever worked with a GPT-4, you may have been at the time of examining the bill and think, “How did this higher?!” Each of the API driving you do is eat tokens, which directly affect the lattens and costs. But without being taken away, you don't have a view to how they are used or how to do it.

That's where Langsmith Comes in. It is not just following your LLM calls but also allows you to log in, monitor, and see the use of Token to all the steps in your transit. In this guide, we will cover:

1. Why should you follow the news?
2. How can trees set up?
3. Can you imagine the use of Token in LangSmith Dashboard?

Obvious Why should you follow the news?

News of changing tokens because all partnerships with a larger language model has a direct currency tied to the number of tokens, both input and the results of the model. Without monitoring, a small amount of employment in Prompts, unnecessary context, or unnecessary applications can make your money quiet and reduce the performance.

By following the tokens, you get visibility where eaten. In this way you can remove the movement, associated work flow, and the maintenance of costs. For example, if your Chatbot uses 1,500 tokens for each application, it decreases that 800 tokens can cut costs about half. The idea of ​​following the Token trail is somewhat active:

Obvious Setting up LangSmith to sign in token

// Step 1: Enter the required packages

pip3 install langchain langsmith transformers accelerate langchain_community

// Step 2: Do all the required imports

import os
from transformers import pipeline
from langchain.llms import HuggingFacePipeline
from langchain.prompts import PromptTemplate
from langchain.chains import LLMChain
from langsmith import traceable

// Step 3: Prepare Langsmith

Set your API key and project name:

# Replace with your API key
os.environ["LANGCHAIN_API_KEY"] = "your-api-key"
os.environ["LANGCHAIN_PROJECT"] = "HF_FLAN_T5_Base_Demo"
os.environ["LANGCHAIN_TRACING_V2"] = "true"


# Optional: disable tokenizer parallelism warnings
os.environ["TOKENIZERS_PARALLELISM"] = "false"

// Step 4: Upload Hugging Focus Model

Use a CPU friend model like Google / Flan-T5-Base And allow the more natural outline sample:

model_name = "google/flan-t5-base"
pipe = pipeline(
   "text2text-generation",
   model=model_name,
   tokenizer=model_name,
   device=-1,      # CPU
   max_new_tokens=60,
   do_sample=True, # enable sampling
   temperature=0.7
)
llm = HuggingFacePipeline(pipeline=pipe)

// Step 5: Create Prompt and Chain

Describe a fast template and connect it with your variable face pipes using LLMCHAIN:

prompt_template = PromptTemplate.from_template(
   "Explain gravity to a 10-year-old in about 20 words using a fun analogy."
)


chain = LLMChain(llm=llm, prompt=prompt_template)

// Step 6: Do the work and download with Langsmith

Use @tragegiable ornaments to write automatically, output, use of Token, and time of operation:

@traceable(name="HF Explain Gravity")
def explain_gravity():
   return chain.run({})

// Step 7: Get started work and print results

answer = explain_gravity()
print("n=== Hugging Face Model Answer ===")
print(answer)

Which is output:

=== Hugging Face Model Answer ===
Gravity is a measure of mass of an object.

// Step 8: Check LangSmith Dashboard

Go to Smith.langchain.com → Following projects. You will do this:

You can also see costs associated with each project, allowing you to analyze your billing. Now see the use of tokens and other understanding, click on your project. And you will see:

The red box highlights and calculates the number of runs you made on your project. Click on any running and you will see:

You can see different things here such as perfect tokens, latency, etc. Click Description as shown below:

You can now look at the graphs later to track the styles of the use of Token, check the latency rating for each application, compare to enter the installation tokens, and point to use times. This understanding helps increase the increase, manage costs, and developing the functioning of the model.

Please cross down to watch all graphs associated with your project.

// Step 9: Check LangSmith Dashboard

You can analyze plenty of understanding like:

• View the exemplary traces: Click Trace to see a detailed execution, including raw inserts, produced produced, and the metric operations
• Check each track: Each track, you can view all the murder action, seeing the motives, outgoing, use of Token and Latency
• See the use of Token & Latency: Prices with detailed tokens and processing periods help identify bottles and function effectively
• Check Chains: Use LangSmith Assessment Tools to check out conditions, track model, and compare out
• TEST IN THE MARRANCE STATE: Correct the parameters such as temperatures, fast templates, or sample settings to function properly your model

With this tip, you now have a full look of your covered face model, use of Token, and complete operation in LangSmith Dashboard.

Obvious How can you tell and adjust the tokens tokens?

Once you have found logged, you can:

• See if it would be very remembered
• Identify Calls when the model produces excessive
• Switch to small models in cheap jobs
• Copying Answers to avoid twiceplicit requests

This is the golden to correct long chains or agents. Get a step that consumes multiple tokens and correct it.

Obvious Rolling up

This is how you can set and use LangSmith. Token login installation is not just money to save, it's about creative creativity, well-effective apps. This guide provides a basis, you can learn more about testing, check, and analyze your own work flow.

Kanal Mehreen Are the engineering engineer and a technological author interested in the biggest interest of data science and a medication of Ai and medication. Authorized EBOOK “that added a product with chatGPT”. As a Google scene 2022 in the Apac, it is a sign of diversity and the beauty of education. He was recognized as a Teradata variation in a Tech scholar, Mitacs Globalk scholar research, and the Harvard of Code Scholar. Kanalal is a zealous attorney for a change, who removes Femcodes to equip women to women.