To create a two-language interview interface with an Arabic Source of Meraj-Mini by Arcee Ai: PHTTORCH, Transformers, accelerated, and gradio
In this lesson, we use the Assist for two of the Arcee's Meraj-Minij-mini model, sent to September on Google Colab using T4 GPU. This lesson illustrates the skills of open languages while providing practical, signs in spreading AI solutions within free free resources issues. We will use a powerful stack of tools including:
- Arcee Memori Model
- Transformers Library Uploading Model and Tokobation
- Accelerates with appropriate value bitsandbytes
- The deeper design pytro of learning
- Gradio to build an active web interface
# Enable GPU acceleration
!nvidia-smi --query-gpu=name,memory.total --format=csv
# Install dependencies
!pip install -qU transformers accelerate bitsandbytes
!pip install -q gradioFirst we enable GPU rushing by asking GPU's name and complete memory using the Nvidi-SMI command. Then it installs and renews the important libraries of Python – such as converts, accelerates, bilsandbytes, and gradio-funding services and sending apps.
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM, pipeline, BitsAndBytesConfig
quant_config = BitsAndBytesConfig(
load_in_4bit=True,
bnb_4bit_quant_type="nf4",
bnb_4bit_compute_dtype=torch.float16,
bnb_4bit_use_double_quant=True
)
model = AutoModelForCausalLM.from_pretrained(
"arcee-ai/Meraj-Mini",
quantization_config=quant_config,
device_map="auto"
)
tokenizer = AutoTokenizer.from_pretrained("arcee-ai/Meraj-Mini")Then we prepare for 4-bit quality settings using a BssandbytbyConfig Uploading modeling model, and loading the “Arceue-AI / Meraj-mini model” from its map device, automatically functioning.
chat_pipeline = pipeline(
"text-generation",
model=model,
tokenizer=tokenizer,
max_new_tokens=512,
temperature=0.7,
top_p=0.9,
repetition_penalty=1.1,
do_sample=True
)
Here we build a pipeline prepared for discussion using the facial pipeline. Preparing for new tokens, temperature, top_p, and multiplication fee to measure diversity and meet during text.
def format_chat(messages):
prompt = ""
for msg in messages:
prompt += f"<|im_start|>{msg['role']}n{msg['content']}<|im_end|>n"
prompt += "<|im_start|>assistantn"
return prompt
def generate_response(user_input, history=[]):
history.append({"role": "user", "content": user_input})
formatted_prompt = format_chat(history)
output = chat_pipeline(formatted_prompt)[0]['generated_text']
assistant_response = output.split("<|im_start|>assistantn")[-1].split("<|im_end|>")[0]
history.append({"role": "assistant", "content": assistant_response})
return assistant_response, historyWe explain two jobs to facilitate variable communication. The first work formatted conversation history with customized associates and customer derives, and you are generating the new user message, and produces the answer using the storage pipe, and renew the chat history accordingly.
import gradio as gr
with gr.Blocks() as demo:
chatbot = gr.Chatbot()
msg = gr.Textbox(label="Message")
clear = gr.Button("Clear History")
def respond(message, chat_history):
response, _ = generate_response(message, chat_history.copy())
return response, chat_history + [(message, response)]
msg.submit(respond, [msg, chatbot], [msg, chatbot])
clear.click(lambda: None, None, chatbot, queue=False)
demo.launch(share=True)Finally, we create an interface of the page based chatbot user using Gradio. Creates UI architectural components of chat historical, message, and clear history button, and explains the response to the generation pipeline to renew conversation. Finally, shortly introduced to the partnership enabled for the public access.
Here is the Colab Notebook. Also, don't forget to follow Sane and join ours Telegraph station including LinkedIn Grtopic. Don't forget to join ours 80k + ml subreddit.
Asphazzaq is a Markteach Media Inc. According to a View Business and Developer, Asifi is committed to integrating a good social intelligence. His latest attempt is launched by the launch of the chemistrylife plan for an intelligence, MarktechPost, a devastating intimate practice of a machine learning and deep learning issues that are clearly and easily understood. The platform is adhering to more than two million moon visits, indicating its popularity between the audience.
Parlint: Create faithful AI customers facing agents with llms 💬 ✅ (encouraged)
