Send QWen models with Amazon Bedrock Custom Model Veer

We are happy to announce that Amazon Bedrock Custom Model Model Section now supports QWEN models. You now enter the custom QWEN2, QWEN2_VL, and QWEN2_VL of construction, including models such as accumulated in accordance with SQUPH. 2.5 VL, and QWQ 32b. You can bring your QWEN models to Amazon Bedrock and use them in a fully owned place, advance – except manage infrastructure or ministerial model.

In this post, we cover how to send QWEN 2.5 models by Amazon Bedrock Custom Model, making them accessible to the use of Ai-ar Entrouairs.

Views of QWEN models

Legal 2 and 2,5 are the families of large language models, are found in various types of size and special variations to comply with different requirements:

• Models of common language: Models range from 0.5B to 72b parameters, with both versions and teach types of general purposes
• QWEN 2.5-Coder: Special coding and completion
• QWEN 2.5-MATT: Focus on the improvements of mathematical calculations
• QWEN 2.5-VL (Language of Vision): Piggency processing and videos, enabling multimodal apps

Amazon Bedrock Custom Model Model

Amazon Bedrock Custom Model matches Importing and Use of Your Basic Models and the existing Basis Models (FMS) with one unemployment, united. You can access your import models and without the need to manage underground infrastructure. Prompt the development of your AI application by combining your customer service models with the lowly amazon bedrock tools and the basics of Amazon Bedrock, and Amazon Bedrock Guarteraages, and Amazon Bedrock Agents. Amazon Bedrock Custom Model Loction is often found in US-East (N. Virginia, in US-West (Oregon), and Europe (Frankfurt) AWS AWS AWS. Now, we will examine how you can use QWEN 2.5 two common cases used: as a customer's assistant and the understanding of the image. QWEN2.5-Coder is a secluype of the Art-of-The-The-The-The-The-Art, Similar Skills Like GPT-4O. It sponsors more than 90 editing languages ​​and passes at the level of the code, correcting, and consultation. QWEN 2.5-VL brings developed of multimodal skills. According to QWEN, QWEN 2,5-VL is not only able to see things such as flowers and animals, but also to analyzing charts, scriptures from pictures, and processing long videos.

Requirements

Before importing a QWEN model with Amazon Bedrock Custom Model, make sure you have the next place:

1. An active account of the times
2. Amazon Simple Storage Service (Amazon S3) Last Bucket of QWEN Model files
3. Permissions to be sufficient for building a model of Amazon Bedrock model
4. It is guaranteed that your circuit supports Amazon Bedrock Custom Model Model

Apply Case 1: QWEN Codes Assistant

In this example, we will show how we can create a helper that using QWEN2.5-Coder-7b-tree model

1. Go to the face of face and search and copy the QWEN / QWEN2.5-code-6b-strate id:

Will use Qwen/Qwen2.5-Coder-7B-Instruct all rest. We don't show good listening steps, but you may also go well before importing.

2. Use the following command to download a summary of your sibling. The Python Library of the District Convention provides the Snapshot Download of the Snapshot Download:

from huggingface_hub import snapshot_download

snapshot_download(repo_id=" Qwen/Qwen2.5-Coder-7B-Instruct", 
                local_dir=f"./extractedmodel/")

According to your model size, this can take a few minutes. When completing, your QWen Coteer Coteer 7b folder will contain the following files.

• Configuration files: Including config.json, generation_config.json, tokenizer_config.json, tokenizer.jsonbeside vocab.json
• Model files: Four safetensor files and model.safetensors.index.json
• Thought: LICENSE, README.mdbeside merges.txt

3. Enter model in Amazon S3, using boto3 or command line:

aws s3 cp ./extractedfolder s3://yourbucket/path/ --recursive

4. Start the function of the import model using the next API call:

response = self.bedrock_client.create_model_import_job(
                jobName="uniquejobname",
                importedModelName="uniquemodelname",
                roleArn="fullrolearn",
                modelDataSource={
                    's3DataSource': {
                        's3Uri': "s3://yourbucket/path/"
                    }
                }
            )
            

You can also do this using Amazon Bedrock ASZON Management Management.

5. In Amazon Bedrock Console, Select Models are imported to the wavering pane.
6. Designate Import the model.

7. Enter information, including a The model name, Import the work nameand model S3 area.

8. Create a new role of the service or use the existing service role. And select the import model

9. After selecting Import In Console, you must see the status as an import model:

If you use your role, make sure you add the following relentory relationships as you focus the role of the model service.

After your model is taken, wait for the model access to be ready, and then discuss the model on the playground or API. In the next example, we include it Python Moving the model to direct Python code to judge items in the S3 bucket. Remember to use the relative installation template template in the required format. For example, you can find the appropriate chat template of any compatible model in the surface of the face below:

from transformers import AutoTokenizer
tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-Coder-7B-Instruct")

# Instead of using model.chat(), we directly use model.generate()
# But you need to use tokenizer.apply_chat_template() to format your inputs as shown below
prompt = "Write sample boto3 python code to list files in a bucket stored in the variable `my_bucket`"
messages = [
    {"role": "system", "content": "You are a helpful coding assistant."},
    {"role": "user", "content": prompt}
]
text = tokenizer.apply_chat_template(
    messages,
    tokenize=False,
    add_generation_prompt=True
)

Note that when using invoke_model APIs, you should use the full Amazon Resource (Arn) name (Arn) of the imported model. You can find the model arn at the Bedrock Console, by wandering the maximum models and view the model details page, as shown in the following

After model is ready to be humble, you can use the Bedrock Console Chatroom or API for requesting model.

Apply case 2: QWWEN 2.5 VL Image Understanding

QWEN2.5-VL- * provides a lot of multimodol energy, consolidating the idea and understanding of languages ​​in one model. This section shows how we can send QWEN2.5-VL through the Amazon Bedrock Custom Model and test its image understanding skills.

Import the QWEN2.5-VL-7B to Amazon Bedrock

Download model from the baggingface face and upload it to Amazon S3:

from huggingface_hub import snapshot_download

hf_model_id = "Qwen/Qwen2.5-VL-7B-Instruct"

# Enable faster downloads
os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1"

# Download model locally
snapshot_download(repo_id=hf_model_id, local_dir=f"./{local_directory}")

Next, submit the Amazon Bedrock model (either with a console or API):

response = bedrock.create_model_import_job(
    jobName=job_name,
    importedModelName=imported_model_name,
    roleArn=role_arn,
    modelDataSource={
        's3DataSource': {
            's3Uri': s3_uri
        }
    }
)

Test the skills to see

After the import is complete, test the model by the photograph. QWEN2.5-VL- * Requires to format appropriate formatting:

def generate_vl(messages, image_base64, temperature=0.3, max_tokens=4096, top_p=0.9):
    processor = AutoProcessor.from_pretrained("Qwen/QVQ-72B-Preview")
    prompt = processor.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
    
    response = client.invoke_model(
        modelId=model_id,
        body=json.dumps({
            'prompt': prompt,
            'temperature': temperature,
            'max_gen_len': max_tokens,
            'top_p': top_p,
            'images': [image_base64]
        }),
        accept="application/json",
        contentType="application/json"
    )
    
    return json.loads(response['body'].read().decode('utf-8'))

# Using the model with an image
file_path = "cat_image.jpg"
base64_data = image_to_base64(file_path)

messages = [
    {
        "role": "user",
        "content": [
            {"image": base64_data},
            {"text": "Describe this image."}
        ]
    }
]

response = generate_vl(messages, base64_data)

# Print response
print("Model Response:")
if 'choices' in response:
    print(response['choices'][0]['text'])
elif 'outputs' in response:
    print(response['outputs'][0]['text'])
else:
    print(response)
    

When provided by the Kati Kati Scriptural example. This reflects QWEN2.5-VL- * of a model to process visual information and generate the descriptions of appropriate documentation.

Model response:

This image features a close-up of a cat lying down on a soft, textured surface, likely a couch or a bed. The cat has a tabby coat with a mix of dark and light brown fur, and its eyes are a striking green with vertical pupils, giving it a captivating look. The cat's whiskers are prominent and extend outward from its face, adding to the detailed texture of the image. The background is softly blurred, suggesting a cozy indoor setting with some furniture and possibly a window letting in natural light. The overall atmosphere of the image is warm and serene, highlighting the cat's relaxed and content demeanor. 

Putting higher property

You can use the Amazon Bedrock Custom Model to use your custom model metals within Amazon Bedrock for amazon facilities and the Amazon Bedrock Hodd FMS according to the managed way. Importing a custom model no one is to import the model. You are charged for detection based on two factors: the number of copies of active models and their work period. The billing occurs in 5 minutes increase, from reducing the effectiveness of each copy of the model. Each price is each of each copy of the projects involved in the construction, Length, region, and compute Unit Version, and taken by the Model Copy size. The custom model includes deportation depends on the model forming model, parameter calculation, and format. Automatic Amazon Bedrock regulates rating based on your use patterns. If there is no appeal for 5 minutes, it is estimated in zero and in the basic time when needed, or this can include cold cold latency up to a minute. Additional copies are added when the volume of speculation consistently exceeds one single concute corrections. High opposition and per copy is determined during importing, based on items such as filing / removing tone

For more information, see Amazon Bedrock prices.

Clean

To avoid ongoing cases after completing the exams:

1. Remove your qwen models from Amazon Bedrock Custom Model Templete using a console or API.
2. Optionally delete model files from your S3 bucket if you no longer need.

Remember that while Amazon Bedrock Custom Model Model does not charge the actual import process, you are charged for use of the model and maintenance.

Store

Amazon Bedrock Custom Model Templevol gives organizations using public models like NaQn1,5, among others, while benefiting from the Enterprise Grade Infrastructure. The unexpected Amazon Bedrock environment removes the difficulties of managing the submission of models and operations, which allows the parties to focus on the construction systems instead of infrastructure. With factors such as the default measurement, prices for use, and seamlessly compilation with AWS services, Amazon Bedrock provides a complement to the production of AI. The combination of QWEN 2.5'S Advanced A Accing and Amazon Bedrock infrastructure is managed by providing proper balance, expenses, and effective. Organizations may start in small models and reach top, while storing full control over the post of models and benefits to the AWS Security and compliance skills.

For more information, see the Amazon Bedrock user guide.

About the authors

Ajit MahedDy An experienced product and the Go-to-Market leader (GTM) for more than 20 years of experience in the management of product, engineering and market. Before having his current role, Ajit product formation has led AI / ML products to leading technical companies, including Uber, Thuring, and eHealth. You are interested in the technological technological technology and driving the actual world impact with Ai Generative AI.

Subramannian Shreyas You are the main data scientist and helping customers through the productive AI and deep reading to resolve their business challenges using AWS services. Shrewas has a source of great performance and doing well and the use of ML and the validity of the learning of accelerating jobs.

Yanyan Zhang He is a UI scientist in Ai in Amazon Web Services in Amazon Web Services, where he worked in AI / ML technology as a fatal AI specialist, who helped customers use the Generative AI to fulfill the necessary results. Yanyan graduated from Texas A & M University with a PhD in electrical engineer. Outside work, he loves to travel, work, and examining new things.

Dharinee Gupta Is the AWS manager of AWROCK, where they focus on making customers enables easy source models using incoming solutions. Her team is most effective in doing well these models to bring the best balance of costs to customers. Before his current role, he found a broader experience of approval and programs for Amazon approval, which improves solutions for safe access to amazon contributions. Dharinee likes to make advanced AI advanced technology accessible and work with AWS customers.

Lokewaran ravi Do the main engineer of the deepest learning of AW Aw Aw Aw, looking forward to ML performance, modeling of model, and AI safety. It focuses on developing efficiency, costs, and safe environmental conditions in Demornandazaze Ai Technologies, which makes ML determination and contributed to all industries.

JUNE win The main product manager with Amazon Sagemaker JumpStart. Focused on making easily accessible and useful base models and benefit customers to issue AI services apps. His experience in Amazon also includes apps for mobile phones and delivery of the last miles.