GPT OSS models from Opelai now are available on Sagemaker Jumpstart

Today, we are very happy to announce the availability of Open Open Open Open Open Open OsS OSS, gpt-oss-120b including gpt-oss-20bfrom Opelai in Amazon Sagemaker Jumpstart. At this time opening, you can now send back back models in the background construction, trying, and to balance your productive AI ideas with AWS.

In this post, we show how we can start with these types of Sagemaker JumpStart.

Looking for everything

Opelai GPT OSS (gpt-oss-120b including gpt-oss-20b) Excel by filing codes, science analysis, and mathematical consultation activities. Both models include the 128K context window and the moderate (low / high / higher) context. They support the integration of external tools and can be used in the travel of agentic services through the structures such as strands agents, open source of AI agent SDK. With full-thought-out thinking skills, you get detailed visibility in the model consultation process. You can use Opelai SDK to call your Sagemaker in the exact endpoint simply by renovating the conclusion. Models provide you with a variable of changing and customizing your specific business needs while benefiting from business grade and seamless growth.

SAGUKMAKER JUMPSTART A fully-owned service provides various models of state-of-the-art Foundation (FMS) various cases that are used as a content, writing the code, to answer, separators, and recovery. It provides a collection of previously-trained models to send, accelerate the development and submission of the requests of the study machine (ML). One of the important Sagemaker JumpStart stuff is a model model, providing a large catalog of former professional models, such as Opelai, with various functions.

You can now find and submit Openai Models in Amazon Sagemaker Studio or in order with Amazon Sagemaker Python SDK, preparation for model and MLOS controls with Amazon Sagemaker, or amazon logs. Models are shipped in a safe AWS location and under your VPC controls, to help support the safety of the business requirements for business security.

You can find GPT OSS models from the US East (Ohio, N. Virginia) and Asia Pacific (Mumbai, Tokyo) AWS regions.

Throughout this example, we use gpt-oss-120b the model. These steps can be repeated with gpt-oss-20b the model again.

Requirements

Sending GPT OSS models, you must have the following requirements:

• AWS account that will contain your AWS services.
• The Role of AWS IDentity and Access Management (IAM) to access the Sagemaker. To learn more about how Iam works with the Sagemaker, see the AWS ownership and ASZON SAGENAKER ACT.
• Sagemaker studio, example of the Sagemaker Patebook, or a collision development (right) such as Pycharm or Visual Studio. We recommend using Sagemaker Studio for Shipping and Direct Tukes.
• Sending GPT OSS models, make sure you have access to different types of model. You can find these Insurance recommendations on the Sagemaker JumpStart model card. The default type of both these models is P5.48xlage, but you can use other P5's sites where available. To ensure that you have the required service ratings, complete the following steps:
  • In the Console Service Embassy, less AWS servicesselect Amazon Sagemaker.
  • Check that you have enough rate of a typical example required for the end of the end.
  • Make sure at least one of these Instrens species is available in your target region.
  • If needed, request a quota increase and contact your AWS account group.

Send GPT-OSS-120B with Sagemaker Jumpstart Ui

Complete these next steps of use gpt-oss-120b With Sagemaker JumpStart:

1. In Sagemaker Console, Select Favorite to the wavering pane.
2. The first time users will be notified that they create a domain. If not, select Open the studio.
3. In the Studio Studio Studio, Sagemaker JumpStart Access to Choose Bend to the wavering pane.
4. On the Sagemaker JumpStart's arrival page, search gpt-oss-120b using the search box.

5. Select a model card to view details about the model such as license, the data used for training, and how to use the model. Before you can use the model, review your appointment and model from the model card. The model details page includes the following information:
   1. The model name and information of the provider.
   2. A Deploy button to send the model.

6. Designate Deploy to continue the shipment.
   1. A Members NAME EDPOINTEnter the endpoint name (up to alphanumeric characters).
   2. A Members Number of conditionsEnter the number between 1-100 (default: 1).
   3. A Members Type TypeChoose your example. Working well with gpt-oss-120bGPU type based on GPU similar to P5.48xgarage is recommended.

7. Designate Deploy To send the model and create a conclusion.

When the shipment is completed, your EDPONT status will change Setting a Center Insice. In the meantime, the model is ready to accept applications for dignity through the conclusion. When the shipment is completed, you can ask the model using the Sagemaker Runtime client and integrate your apps.

Send GPT-OSS-120B with Sagemaker Python SDK

Shiping Using SDK, Start by selecting the gpt-oss-120b the model, described by the model_id perpetual openai-reasoning-gpt-oss-120b. You can send your choice of model to the Sagemaker using Python SDK examples in the following sections. Similarly, you can answer gpt-oss-20b using its model ID.

Enable web search to your model with EXA

Automatically, Sagemaker Jumpstart Run models in separation network. GPT OSS models come with a built-in search tool using EXA, an API screening based web-based Web site enabled by the eMbeddings. In order to use this tool, Openai requires customers earned API key from EXA and exceeds the key as the variety of nature JumpStartModel An example when we use it using Sagemaker Python SDK. The following code for the model use of the modemaker with network is disabled and exceeded the EXA API key to the model:

from sagemaker.jumpstart.model import JumpStartModel 

accept_eula = True 
model = JumpStartModel(
    model_id="openai-reasoning-gpt-oss-120b",
    enable_network_isolation=False, 
    env={
        "EXA_API_KEY": ""
    }
) 
predictor = model.deploy(
    accept_eula=accept_eula
)

You can change this configuration by specifying some non-unique prices for JumpStartModel. User License Agreement (EULA) must be explicitly defined as True accepting the terms. For the preceding posture, because network contract is divided into the shipping period, we do not want to create a new destiny.

Optionally, you can send your model to default jumpstart prices (with a solse network enabled) as follows:

from sagemaker.jumpstart.model import JumpStartModel 
accept_eula = True 
model = JumpStartModel(model_id="openai-reasoning-gpt-oss-120b") 
predictor = model.deploy(accept_eula=accept_eula)

Run Reception With Sagemaker Predictor

After model is transferred, you can run against the end of the Sagemaker feed:

payload = {
    "model": "/opt/ml/model",
    "input": [
        {
            "role": "system",
            "content": "You are a good AI assistant"
        },
        {
            "role": "user",
            "content": "Hello, how is it going?"
        }
    ],
    "max_output_tokens": 200,
    "stream": "false",
    "temperature": 0.7,
    "top_p": 1
}
    
response = predictor.predict(payload)
print(response['output'][-1]['content'][0]['text'])

We receive the following reply:

Hey there! All good on my end—just ready to dive into whatever you need. How’s it going on your side?

To call

GPT OSS models are trained in responding format by describing conversation structures, producing a consultation effect on planning telecommunications. The format is designed to imitate Opelai API answers, so if you use the api before, this form should hope it is familiar to you. The model should not be used without using the agreement format. The following example reflect the example of the use of the tool of this format:

payload= {
  "model": "/opt/ml/model",
  "input": "System: You are ChatGPT, a large language model trained by OpenAI.nKnowledge cutoff: 2024-06nCurrent date: 2024-08-05nnreasoning: mediumnn# Valid channels: analysis, commentary, final. Channel must be included for every message.nCalls to these tools must go to the commentary channel: 'functions'.nn# Toolsnn## functionsnnnamespace functions {nn// Gets the current weather for a specific location.ntype get_current_weather = (_: {n// The city and state/country, e.g. "San Francisco, CA" or "London, UK"nlocation: string,n// Temperature unit preferencenunit?: "celsius" | "fahrenheit", // default: celsiusn}) => any;nn} // namespace functionsnnDeveloper: You are a helpful AI assistant. Provide clear, concise, and helpful responses.nnHuman: What's the weather like in Seattle?nnAssistant:",
  "instructions": "You are a helpful AI assistant. Provide clear, concise, and helpful responses.",
  "max_output_tokens": 2048,
  "stream": "false",
  "temperature": 0.7,
  "reasoning": {
    "effort": "medium"
  },
  "tools": [
    {
      "type": "function",
      "name": "get_current_weather",
      "description": "Gets the current weather for a specific location",
      "parameters": {
        "type": "object",
        "properties": {
          "location": {
            "type": "string",
            "description": "The city and state/country, e.g. 'San Francisco, CA' or 'London, UK'"
          },
          "unit": {
            "type": "string",
            "enum": ["celsius", "fahrenheit"],
            "default": "celsius",
            "description": "Temperature unit preference"
          }
        },
        "required": ["location"]
      }
    }
  ],
}

We receive the following reply:

{'arguments': '{"location":"Seattle, WA"}', 'call_id': 'call_596a67599df2465495fd444772ff9539', 'name': 'get_current_weather', 'type': 'function_call', 'id': 'ft_596a67599df2465495fd444772ff9539', 'status': None}

Clean

After you have finished using a letter of writing, be sure to remove resources you created to avoid more charge. For more information, see Delete Endpoints and resources.

predictor.delete_model()
predictor.delete_endpoint()

Store

In this post, show how we can send and start with Opelai Os Models (gpt-oss-120bincluding gpt-oss-20b) In Sagemaker Jumptart. These consultation models bring advanced artificial skills, science analysis, and statistical tasks directly to your AWS environment and business grade and business distance and disability.

Try new models, and share your feedback to a comment.

About the authors

Pradyun RamadoraiTop Engineers Developing Software
Malav ShastrAtriSoftware Development Engineers
Varun MorimettySoftware Development Engineers
Evan KravitzSoftware Development Engineers
Benjamin CrabtreeSoftware Development Engineers
Shen TengSoftware Development Engineers
Loki RaviTop Engineers Developing Software
Unithan VijeaderanSpecialist Solutions Builder
Breanne WarnerEnterprise Solutions builder
Yotam mossSoftware Development Manager
Mike JamesSoftware Development Manager
Sadaf FardeenSoftware Development Manage
Siddharth ShahThe main software engineer
JUNE winPrimary product manager