5 tips for construction prepared for transform pipes

Photo for Editor | Chatgt

Obvious Introduction

Kisses face He has become a standard for many AI and the data scientists because it drastically reduces a barrier to an advanced AI. Instead of working with AI models from the beginning, enhancements can reach a variety of beautiful models without suffering. Users can synchronize with these models with custom datasets and to appreciate faster.

One of the hugging face structures are the ones Transformers' pipesThe series of packages containing a non-eligible model, its Tokenzer, before and after working, and related issues to use AI. These are mysterious code pipes and give a simple, seamless API.

However, operating on transformers pipes can be defiling and may not appear in the relevant pipeline. That is why we will explore various five ways, you can create your TransformMers pipes.

Let's get into it.

Obvious 1. Batch Endence Applications

Usually, when using transformers pipes, we do not fully implement the Graphics Proping Unit (GPU). The batch functionality of a lot of installation can increase the use of GPU and improve infance function.

Instead of processing one sample at a time, you can use pipes batch_size Parameter or exceeded the installation list to model processing several inputs of one. Here is an example of the code:

from transformers import pipeline

pipe = pipeline(
    task="text-classification",
    model="distilbert-base-uncased-finetuned-sst-2-english",
    device_map="auto"
)

texts = [
    "Great product and fast delivery!",
    "The UI is confusing and slow.",
    "Support resolved my issue quickly.",
    "Not worth the price."
]

results = pipe(texts, batch_size=16, truncation=True, padding=True)
for r in results:
    print(r)

By working hard for requests, you can get the highest filling of the smallest impact on latency.

Obvious 2. Use lower and quantity

Many good models failed failed because progress and production facilities do not have enough memory. The accuracy of the low price helps reduce memory usage and accelerate humility without giving up the greater accuracy.

For example, here you can use half-specifics in GPU on Transform Pipeline:

import torch
from transformers import AutoModelForSequenceClassification

model = AutoModelForSequenceClassification.from_pretrained(
    model_id,
    torch_dtype=torch.float16
)

Similarly, ways to reduce energy may oppress the metals of model without a visual deterioration:

# Requires bitsandbytes for 8-bit quantization
from transformers import AutoModelForCausalLM

model = AutoModelForCausalLM.from_pretrained(
    model_id,
    load_in_8bit=True,
    device_map="auto"
)

A lower and quantity of the production is used to accelerate the pipeline and reduce the use of memory without a significant impact on model.

Obvious 3. Choose Model Properties

In many programs, you don't need too much model to solve work. Choosing the construction of transter transformer, such as a sound model, usually better latency and passing accuracy.

Types of integrated types or filter types, such as distilbert, keep most of the accuracy of the original model but of the few parameters, which leads to fast access.

Choose her construction model for influence and fits the accuracy requirements for your work.

Obvious 4. Find a temporary maintenance

Multiple Cute Shuthu programs by multiplying expensive work. The boat maintenance can significantly improve working through the effectiveness of expensive integration results.

with torch.inference_mode():
    output_ids = model.generate(
        **inputs,
        max_new_tokens=120,
        do_sample=False,
        use_cache=True
    )

Active preservation reduces the time of integrating and enhances respondent times, reducing latency in production systems.

Obvious 5. Use time to speed up Optimum (Onx Runtime)

Many pipes run a Pytorch Well, the appropriate mode, add a python to more memory copies. Use Get well With Open Neal Network Exchange Exchange (Onx) Runtime – with Onnx Runtime – Change the model into a TULI graph and FUSIs jobs, so the running time can use speedy heads in the use unit (CPU) or GPU has a lower top. The result is often quickly decreasing, especially on the CPU or with mixed hardware, without changing the way you call you pipe.

Enter the required packages with:

pip install -U transformers optimum[onnxruntime] onnxruntime

Then, modify model in such code:

from optimum.onnxruntime import ORTModelForSequenceClassification

ort_model = ORTModelForSequenceClassification.from_pretrained(
    model_id,
    from_transformers=True
)

By turning a pipe into an onnx operation permit, you can save your existing pipe code while you receive low latency and practical tendency.

Obvious Rolling up

Transformers' pipes is the wrapper of the API side in the face system that asks the development of AI application by following the complex code of simple areas. In this article, we examined five tips for the expansion of FACE Transformers, from the batch aversifers pipes, selecting the construction of a well-efficient model, temporarily.

I hope this has helped!

Cornellius Yudha Wijaya It is a scientific science manager and the database author. While working full-time in Allianz Indonesia, she likes to share the python and data advice with social media and media writing. Cornellius writes to a variety of AI and a study machine.