Open a cost-effective AI index using Amazon Bedrock's serverless capabilities with Amazon SageMaker's trained model

In this post, I'll show you how to use Amazon Bedrock—with a fully managed, on-demand API—with your trained or configured Amazon SageMaker model.

Amazon Bedrock is a fully managed service that offers a selection of high performance foundation models (FMs) from leading AI companies such as AI21 Labs, Anthropic, Cohere, Meta, Mistral AI, Stability AI, and Amazon via API one, and a comprehensive set of skills to build productive AI applications in security, privacy, and responsible AI.

Previously, if you wanted to run your custom-tuned models in Amazon Bedrock, you had to manually manage your SageMaker infrastructure or train the models directly within Amazon Bedrock, which required expensive outsourcing.

With Amazon Bedrock Custom Model Import, you can use new or existing models trained or fine-tuned within SageMaker using Amazon SageMaker JumpStart. You can import supported architectures into Amazon Bedrock, allowing you to access them on demand through Amazon Bedrock's fully managed invoke model API.

Solution overview

At the time of writing, Amazon Bedrock supports importing custom models from the following architectures:

• The Mistral
• Flan
• Meta Llama 2 and Llama 3

In this post, we are using the Hugging Face Flan-T5 Base model.

In the following sections, I walk you through the steps to train a model in SageMaker JumpStart and import it into Amazon Bedrock. Then you can interact with your custom model through Amazon Bedrock playgrounds.

What is required

Before you begin, make sure you have an AWS account with access to Amazon SageMaker Studio and Amazon Bedrock.

If you don't already have an instance of SageMaker Studio, see Launch Amazon SageMaker Studio for instructions on creating one.

Train the model in SageMaker JumpStart

Complete the following steps to train the Flan model in SageMaker JumpStart:

1. Open the AWS Management Console and go to SageMaker Studio.

2. In SageMaker Studio, select JumpStart in the navigation pane.

With SageMaker JumpStart, machine learning (ML) professionals can choose from a wide selection of publicly available FMs using pre-built machine learning solutions that can be deployed in a few clicks.

3. Search and select the Face Hugging Foundation Flan-T5

On the model details page, you can review a brief description of the model, how to use it, how to fine-tune it, and what format your training data needs to be in to customize the model.

4. Select The train to start fine-tuning the model to your training data.

Create a training exercise using the default settings. Default fills the training task with the recommended settings.

5. For the example in this post we use an example data set with a large population. If you're using your own data, replace it with The data part, making sure it meets the format requirements.

6. Configure security settings such as AWS Identity and Access Management (IAM) roles, virtual private cloud (VPC), and encryption.
7. Note the value of this Output artifact location (S3 URI) to use later.
8. Submit a job to start training.

You can monitor your work selectively Training you have Activities drop down menu. When the status of the training activity shows as It's finishedthe job is done. With default settings, training takes 10 minutes.

Import the model into Amazon Bedrock

After the model has finished training, you can import it to Amazon Bedrock. Complete the following steps:

1. In the Amazon Bedrock console, select Imported models below Base models in the navigation pane.
2. Select Import the model.

3. Because Model nameenter the known name of your model.
4. Underneath Import settings modelchoose Amazon SageMaker model and select the radio button next to your model.

5. Underneath Access to the servicechoose Create and deploy a new service role and enter the name of the role.
6. Select Import the model.

7. Model import will finish in about 15 minutes.

8. Underneath Playgrounds in the navigation pane, select Text.
9. Select Select a model.

10. Because Sectionchoose Imported models.
11. Because Modelchoose flan-t5-fine-tuned.
12. Because Equipmentchoose On demand.
13. Select Claim.

Now you can interact with your custom model. In the following screenshot, we use our custom model to summarize the description about Amazon Bedrock.

Clean up

Complete the following steps to clean your resources:

1. If you will no longer be using SageMaker, delete your SageMaker domain.
2. If you no longer want to maintain your model artifacts, delete the Amazon Simple Storage Service (Amazon S3) bucket where your model artifacts are stored.
3. To remove your imported model from Amazon Bedrock, at Imported models page in the Amazon Bedrock console, select your model, then select the options menu (three dots) and select Delete.

The conclusion

In this post, we explored how the Import Custom Model feature in Amazon Bedrock enables you to use your trained or fine-tuned custom models to get the most wanted, cost-effective guide. By combining SageMaker's model training capabilities with Amazon Bedrock's fully managed, scalable infrastructure, you now have a seamless way to export your unique models and make them available through a simple API.

Whether you choose the easy-to-use SageMaker Studio console or the flexibility of SageMaker notebooks, you can train and import your models into Amazon Bedrock. This allows you to focus on developing new applications and solutions, without the burden of managing a complex ML infrastructure.

As the power of large modeling languages ​​continues to evolve, the ability to integrate custom models into your applications is becoming increasingly important. With the Amazon Bedrock Custom Model import feature, you can now unlock the full potential of your custom models and deliver a personalized experience to your customers, while benefiting from the scalability and cost effectiveness of a fully managed service.

For a deeper dive into configuring SageMaker, see the Configuring Instructions for FLAN T5 XL with Amazon SageMaker Jumpstart. For more information on Amazon Bedrock, see our Building with Amazon Bedrock workshop.

About the Author

Joseph Sadler is a Senior Solutions Architect on the Global Public Sector team at AWS, specializing in cybersecurity and machine learning. With public and private sector experience, he has expertise in cloud security, artificial intelligence, threat detection, and incident response. His unique background enables him to provide robust, secure solutions that use the latest technology to protect mission-critical systems.