Imemezela ukusekelwa kwe-OpenAI-ehambisanayo ye-API yamaphoyinti okugcina we-Amazon SageMaker AI

Namuhla, i-Amazon SageMaker AI yethula ukusekelwa kwe-OpenAI-ehambisanayo kwe-API kuma-endpoints wesikhathi sangempela. Uma usebenzisa i-OpenAI SDK, i-LangChain, noma i-Strands Agents, manje ungakwazi ukunxenxa amamodeli ku-SageMaker AI ngokushintsha kuphela i-URL yakho yesiphetho. Awulidingi iklayenti langokwezifiso, isembozo se-SigV4, noma ikhodi ebhalwe kabusha.

Uhlolojikelele

Ngalokhu kwethulwa, iziphetho ze-SageMaker AI zidalula i- /openai/v1 indlela eyamukela izicelo zokuqedwa kwengxoxo futhi ibuyisele izimpendulo njengoba ivela esiqukathini, okuhlanganisa nokusakaza-bukhoma. Amaphoyinti okugcina e-OpenAI avuliwe kuwo wonke amaphoyinti okugcina kanye nezingxenye ze-inference kusetshenziswa ama-API ajwayelekile we-SageMaker AI kanye ne-SDK.

Imizila ye-SageMaker AI esekelwe egameni lephoyinti lokugcina ku-URL, ukuze noma yiliphi iklayenti elihambisana ne-OpenAI lisebenza ngaphandle kwebhokisi. Manje ungakha amathokheni omphathi anesikhathi esinqunyelwe wezindawo zakho zokugcina futhi uwasebenzise namakhasimende akho e-OpenAI.

Ukuze uthole isibonelo esisebenzayo esifaka phakathi ukusetshenziswa nokucela, bona incwadi yamanothi ehambisana nayo ku-GitHub.

“Sisebenzisa ama-agent e-AI ekhodi asebenzisa abahlinzeki abaningi be-LLM ngesango le-LLM (i-Bifrost) ekhuluma iphrothokholi yokuqedela ingxoxo ye-OpenAI. Isici sethokheni yomphathi sisivumela ukuthi sengeze i-SageMaker njengendawo yokugcina evumelana ne-OpenAI – akukho ukusayinda kwe-SigV4 ngokwezifiso – ngakho isebenza ngokwemvelo ngesango lethu, i-Vercel AI SDK evamile, kanye ne-Open AI SDK.” kusho uGiorgio Piatti (AI/ML Engineer – Caffeine.AI)

Sebenzisa amacala

Ukugeleza komsebenzi we-Agentic kungqalasizinda ephethwe

Uma wakha ama-agent e-AI anezinyathelo eziningi anezinhlaka ezifana ne-Strands Agents noma i-LangChain, ungakwazi manje ukusebenzisa lokho kugeleza komsebenzi ngokuphelele kuma-endpoints akho e-SageMaker AI. Ama-ejenti akho ashayela amamodeli asebenzisa isixhumi esibonakalayo esivumelana ne-OpenAI ezakhelwe phezu kwawo, kodwa ukucabanga kusebenza ezimweni ezizinikele ze-GPU ku-akhawunti yakho.

Ukusingathwa kwamamodeli amaningi ane-interface eyodwa

Uma usebenzisa amamodeli amaningi—ngokwesibonelo, i-Llama yemisebenzi evamile, i-Mistral ecushwe kahle yomsebenzi oqondene nesizinda, kanye nemodeli encane yokuhlukanisa—ungawasingatha wonke endaweni eyodwa yokugcina ye-SageMaker AI usebenzisa izingxenye ze-inference. Imodeli ngayinye ithola ukwabiwa kwensiza yayo, futhi ngayinye ingabizwa nge-OpenAI SDK efanayo. Awudingi amaklayenti e-API ahlukene noma umzila onengqondo wekhodi yohlelo lokusebenza.

Inikeza amamodeli ashunwe kahle ngaphandle kokushintshwa kwekhodi

Uma ulungisa kahle amamodeli omthombo ovulekile wecala lakho lokusebenzisa elithile, ungawasebenzisa ku-SageMaker AI futhi uwashayele ngesixhumi esibonakalayo esivumelana ne-OpenAI esivele sisetshenziswa izinhlelo zakho zokusebenza. Ushintsho kuphela i-URL yephoyinti lokugcina. Lonke uhlelo lokusebenza—izingcingo ze-SDK, ukuqonda kokusakaza, ukufometha kokwaziswa—akushintshi.

Uhlolojikelele lwesixazululo

Kulokhu okuthunyelwe, sihamba kulokhu okulandelayo:

1. Indlela ukuqinisekiswa kwethokheni yomphathi kusebenza kanjani namaphoyinti okugcina we-SageMaker AI.
2. Isebenzisa futhi ibiza iphoyinti lokugcina lemodeli eyodwa.
3. Isebenzisa futhi inxenxa izingxenye zokushoyo zokusebenzisa amamodeli amaningi.
4. Ukuhlanganiswa nohlaka lwama-Strands Agents.

Okudingekayo

Ukuze uhambisane nalolu hlelo lokusebenza, kufanele ube nalokhu okulandelayo:

• I-akhawunti ye-AWS enezimvume zokudala izindawo zokugcina ze-SageMaker AI.
• I-SageMaker Python SDK (pip install sagemaker).
• I-OpenAI Python SDK (pip install openai).
• Imodeli egcinwe ku-Amazon Simple Storage Service (Amazon S3). Isibonelo, i-Qwen3-4B ilandwe ku-Hugging Face.
• Indima yokwenza i-AWS Identity and Access Management (IAM) ukuze udale izindawo zokugcina, AmazonSageMakerFullAccess inqubomgomo.
• Indima yokwenza i-IAM ne sagemaker:CallWithBearerToken futhi sagemaker:InvokeEndpoint izimvume zokucela indawo yokugcina.

Ukufakazela ubuqiniso ngamathokheni omphathi

I-SageMaker AI OpenAI-izindawo zokugcina ezihambisanayo zisebenzisa ithokheni yomphathi. I-SageMaker Python SDK ihlanganisa ijeneretha yamathokheni edala amathokheni anomkhawulo wesikhathi (avumeleke kufika emahoreni ayi-12) kusuka kuzifakazelo zakho ezikhona ze-AWS. Azikho izimfihlo ezengeziwe noma okhiye be-API abadingekayo.

Ithokheni iqukethe indima yakho noma imininingwane yomsebenzisi, futhi idinga sagemaker:CallWithBearerToken futhi sagemaker:InvokeEndpoint izimvume zesenzo.

Khiqiza ithokheni

Sebenzisa umbhalo olandelayo wePython ukwenza ithokheni.

from sagemaker.core.token_generator import generate_token
from datetime import timedelta

token = generate_token(region="us-west-2", expiry=timedelta(minutes=5))

Ijeneretha yamathokheni isebenzisa noma yiziphi izifakazelo ze-AWS ezitholakala endaweni yakho: izifakazelo zomsebenzisi we-IAM, iphrofayela yesibonelo ku-Amazon Elastic Compute Cloud (Amazon EC2), noma iseshini ye-AWS IAM Identity Center (SSO).

I generate_token umsebenzi ukhiqiza ithokheni yenkampani enesikhathi esinqunyelwe ukuze kuqinisekiswe nge-SageMaker APIs. Ngokuzenzakalelayo, amathokheni asebenza amahora angu-12, nakuba ungakwazi ukukhipha lokhu nge- expiry ipharamitha usebenzisa a timedelta inani noma kuphi phakathi kwesekhondi elingu-1 namahora angu-12. Umsebenzi wamukela isifunda, ongakukhetha aws_credentials_providerkanye nobude besikhathi sokuphelelwa yisikhathi. Uma kungekho Isifunda se-AWS esinikeziwe, sibuyela emuva ku- AWS_REGION imvelo variable. Uma engekho umhlinzeki wokuqinisekisa ohlinzekwayo, uxazulula izifakazelo kusetshenziswa uchungechunge lokuqinisekisa lwe-AWS oluzenzakalelayo, olusesha imithombo eminingi, okuhlanganisa nokuhlukahluka kwemvelo, ~/.aws/credentials, ~/.aws/configimininingwane yesiqukathi, namaphrofayili esibonelo. Ukuze uthole i-oda eligcwele lokulungiswa, bheka imibhalo yemininingwane ye-Boto3.

Amathokheni okuvuselela ngokuzenzakalelayo ezinhlelo zokusebenza ezisebenza isikhathi eside

Kuzinhlelo zokusebenza ezisebenza ngokuqhubekayo, ungasebenzisa iphethini yokuvuselela ngokuzenzakalela usebenzisa httpx ukuze kwenziwe ithokheni entsha esicelweni ngasinye:

import httpx
from sagemaker.core.token_generator import generate_token

class SageMakerAuth(httpx.Auth):
    def __init__(self, region: str):
        self.region = region

    def auth_flow(self, request):
        request.headers["Authorization"] = f"Bearer {generate_token(region=self.region)}"
        yield request

http_client = httpx.Client(auth=SageMakerAuth(region="us-west-2"))

Izimvume ze-IAM

Indima ye-IAM noma umsebenzisi ocela indawo yokugcina udinga izimvume ezilandelayo:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": "sagemaker:InvokeEndpoint",
            "Resource": "arn:aws:sagemaker:::endpoint/"
        },
        {
            "Effect": "Allow",
            "Action": "sagemaker:CallWithBearerToken",
            "Resource": "*"
        }
    ]
}

Njengomkhuba omuhle kakhulu, hlala ubeke imingcele Resource kuma-ARN athile okuphela kwawo InvokeEndpoint esikhundleni sokusebenzisa i-wildcard. Ithokheni yomphathi okhiqizwe kusukela kule ndima inezinga elifanayo lokufinyelela, ngakho inqubomgomo enesikophu esincane ikhawulela irediyasi yokuqhuma uma ithokheni ivezwa ngokungaqondile. Qaphela ukuthi CallWithBearerToken udinga i-wildcard ("*") Okwe Resource inkambu. Ayisekeli imikhawulo yeleveli yensiza.

Indlela ithokheni esebenza ngayo

Ithokheni yomphathi i-base64-encoded SigV4 URL esayinwe ngaphambilini. Uma ufona generate_tokeni-SageMaker AI SDK yenza isicelo kusevisi ye-SageMaker AI CallWithBearerToken isenzo, isayinda endaweni kusetshenziswa izifakazelo zakho ze-AWS, futhi ibhala ngekhodi i-URL ewumphumela esayiniwe njengochungechunge lwethokheni ephathekayo. Ayikho ikholi yenethiwekhi eyenziwa ngesikhathi sokukhiqiza amathokheni. Ukusayina kwenzeka ngokuphelele ohlangothini lweklayenti. Uma wethula leli thokheni endaweni yokugcina ye-SageMaker AI, isevisi iyayikhipha ikhodi, iqinisekise isiginesha ye-SigV4, iqinisekise ukuthi ithokheni ayikaphelelwa yisikhathi, futhi iqinisekisa ukuthi ubunikazi bomsuka be-IAM bunezimvume ezidingekayo. Ukuphila okusebenzayo kwethokheni kungaphansi kwevelu yokuphelelwa yisikhathi kanye nokuba semthethweni okusele kwemininingwane ye-AWS okusetshenziswe ukuyisayina.

Indlela engcono kakhulu yokuphepha: Ithokheni yomphathi iphethe ukugunyazwa okufanayo nemininingwane engaphansi ye-AWS esetshenziselwe ukuyikhiqiza. Phatha amathokheni ngokunakekelwa okufanayo nemininingwane. Ububanzi bendima ye-IAM esetshenziselwa ukukhiqiza amathokheni kuya kuzimvume ezincane ezidingekayo, ngokukhethekile sagemaker:InvokeEndpoint futhi sagemaker:CallWithBearerToken kuma-ARN wephoyinti lokugcina kuphela lowo ofonayo okudingeka awafinyelele. Ungakhiqizi amathokheni ezindimeni ezinezimvume ezinwetshiwe, njengalezo ezinikezwe ngu AdministratorAccess noma SageMakerFullAccess izinqubomgomo eziphethwe.

Ungagcini amathokheni kudiski, ezintweni eziguquguqukayo zemvelo, kumafayela okumisa, kusizindalwazi, noma kuma-caches asakaziwe. Ungawafaki amathokheni, futhi uwadlulisele kuphela ngezivumelwano zokuxhumana ezibethelwe njenge-HTTPS. Ukukhiqiza amathokheni kuwumsebenzi wasendaweni ongenayo inethiwekhi, ngakho-ke umkhuba onconyiwe ukukhiqiza ithokheni entsha lapho usetshenziswa noma usebenzise ukuvuselela okuzenzakalelayo. httpx.Auth iphethini eboniswe esibonelweni esandulele. Lokhu kugwema ubungozi bokuvuza kwamathokheni futhi kukusiza ukuthi usebenzise ithokheni enokuqinisekiswa okuphezulu okusele. Njengomkhuba ongcono kakhulu, setha isikhathi sokuphelelwa yisikhathi kwethokheni sibe yisikhathi esifushane kakhulu esidingwa umsebenzi wakho.

Sebenzisa iphoyinti lokugcina lemodeli eyodwa

Isiphetho semodeli eyodwa sisingatha imodeli eyodwa futhi sinikeza izicelo ngokuqondile. Isibonelo esilandelayo sisebenzisa i-Qwen3-4B kusetshenziswa i-SageMaker AI vLLM Deep Learning Container ku- ml.g6.2xlarge isibonelo.

Qaphela: Amaphoyinti okugcina we-SageMaker AI angenisa izindleko ngenkathi usesevisini, ngokunganaki ithrafikhi. Ukuze uthole imininingwane eyengeziwe, bheka ikhasi lentengo le-Amazon SageMaker AI.

import boto3
import sagemaker
import time
from sagemaker.core.helper.session_helper import Session
from sagemaker.core.helper.session_helper import get_execution_role

# AWS configuration
REGION = "us-west-2"

# Automatically resolve account ID and default SageMaker execution role
session = Session(boto_session=boto3.Session(region_name=REGION))
ACCOUNT_ID = boto3.client("sts", region_name=REGION).get_caller_identity()["Account"]
EXECUTION_ROLE = get_execution_role(sagemaker_session=session)

# HF Model ID
MODEL_HF_ID = "Qwen/Qwen3-4B"

# SageMaker vLLM Deep Learning Container
VLLM_IMAGE = f"763104351884.dkr.ecr.{REGION}.amazonaws.com/vllm:0.20.2-gpu-py312-cu130-ubuntu22.04-sagemaker"

# Instance type (1x NVIDIA L4 GPU)
INSTANCE_TYPE = "ml.g6.2xlarge"

sagemaker_client = boto3.client("sagemaker", region_name=REGION)

print(f"Region: {REGION}")
print(f"Account ID: {ACCOUNT_ID}")
print(f"Execution role: {EXECUTION_ROLE}")
print(f"Model HF ID: {MODEL_HF_ID}")

import time

TIMESTAMP = str(int(time.time()))
SME_MODEL_NAME = f"openai-compat-sme-model-{TIMESTAMP}"
SME_ENDPOINT_CONFIG_NAME = f"openai-compat-sme-epc-{TIMESTAMP}"
SME_ENDPOINT_NAME = f"openai-compat-sme-ep-{TIMESTAMP}"

print(f"Timestamp suffix: {TIMESTAMP}")
print(f"Model: {SME_MODEL_NAME}")
print(f"Endpoint config: {SME_ENDPOINT_CONFIG_NAME}")
print(f"Endpoint: {SME_ENDPOINT_NAME}")

sagemaker_client.create_model(
    ModelName=SME_MODEL_NAME,
    ExecutionRoleArn=EXECUTION_ROLE,
    PrimaryContainer={
        "Image": VLLM_IMAGE,
        "Environment": {
            "HF_MODEL_ID": MODEL_HF_ID,
            "SM_VLLM_TENSOR_PARALLEL_SIZE": "1",
            "SM_VLLM_MAX_NUM_SEQS": "4",
            "SM_VLLM_ENABLE_AUTO_TOOL_CHOICE": "true",
            "SM_VLLM_TOOL_CALL_PARSER": "hermes",
            "SAGEMAKER_ENABLE_LOAD_AWARE": "1",
        },
    },
)
print(f"Model created: {SME_MODEL_NAME}")

sagemaker_client.create_endpoint_config(
    EndpointConfigName=SME_ENDPOINT_CONFIG_NAME,
    ProductionVariants=[
        {
            "VariantName": "variant1",
            "ModelName": SME_MODEL_NAME,
            "InstanceType": INSTANCE_TYPE,
            "InitialInstanceCount": 1,
        }
    ],
)
print(f"Endpoint configuration created: {SME_ENDPOINT_CONFIG_NAME}")

sagemaker_client.create_endpoint(
    EndpointName=SME_ENDPOINT_NAME,
    EndpointConfigName=SME_ENDPOINT_CONFIG_NAME,
)
print(f"Endpoint creation initiated: {SME_ENDPOINT_NAME}")

print("Waiting for endpoint to reach InService status (this takes 5-10 minutes)...")
waiter = sagemaker_client.get_waiter("endpoint_in_service")
waiter.wait(
    EndpointName=SME_ENDPOINT_NAME,
    WaiterConfig={"Delay": 30, "MaxAttempts": 40},
)
print(f"Endpoint is InService: {SME_ENDPOINT_NAME}")

Iphoyinti lokugcina lishintshela ku InService isimo emizuzwini embalwa. Uma isilungile, isebenzisa kokubili i-SageMaker AI evamile /invocations indlela kanye nendlela ehambisana ne-OpenAI ku /openai/v1/chat/completions.

Cela isiphetho semodeli eyodwa

Ngephoyinti lokugcina enkonzweni, linxuse usebenzisa i-OpenAI Python SDK. I-URL eyisisekelo ilandela le fomethi:

https://runtime.sagemaker..amazonaws.com/endpoints//openai/v1

from openai import OpenAI
from sagemaker.core.token_generator import generate_token

REGION = "us-west-2"

sme_base_url = f"

client = OpenAI(
    base_url=sme_base_url,
    api_key=generate_token(region=REGION)
)

print(f"Base URL: {sme_base_url}")

stream = client.chat.completions.create(
    model="",
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Explain how transformers work in machine learning, in three sentences."},
    ],
    stream=True,
)

for chunk in stream:
    if chunk.choices[0].delta.content:
        print(chunk.choices[0].delta.content, end="")
print()

I model insimu idlulela esitsheni. Ngenxa yokuthi i-SageMaker AI icela imizila ngokusekelwe egameni lephoyinti lokugcina ku-URL, ungagcina le nkambu ingenalutho noma uyisethe ukuze ifane negama lemodeli elilindelwe yisitsha sakho.

Sebenzisa iphoyinti lokugcina lengxenye yenkomba

Ngezingxenye ze-inference, ungasingatha amamodeli amaningi endaweni eyodwa yokugcina, ngayinye enokwabiwa kwensiza okuhlanganisiwe okuzinikele. Ngezingxenye ze-inference, imodeli ihlotshaniswa nengxenye esikhundleni sokucushwa kwephoyinti lokugcina:

IC_MODEL_NAME = f"openai-compat-ic-model-{TIMESTAMP}"
IC_ENDPOINT_CONFIG_NAME = f"openai-compat-ic-epc-{TIMESTAMP}"
IC_ENDPOINT_NAME = f"openai-compat-ic-ep-{TIMESTAMP}"
IC_NAME = f"openai-compat-ic-qwen3-4b-{TIMESTAMP}"

print(f"Model: {IC_MODEL_NAME}")
print(f"Endpoint config: {IC_ENDPOINT_CONFIG_NAME}")
print(f"Endpoint: {IC_ENDPOINT_NAME}")
print(f"Inference comp: {IC_NAME}")

sagemaker_client.create_model(
    ModelName=IC_MODEL_NAME,
    ExecutionRoleArn=EXECUTION_ROLE,
    PrimaryContainer={
        "Image": VLLM_IMAGE,
        "Environment": {
            "HF_MODEL_ID": MODEL_HF_ID,
            "SM_VLLM_TENSOR_PARALLEL_SIZE": "1",
            "SM_VLLM_MAX_NUM_SEQS": "4",
            "SM_VLLM_ENABLE_AUTO_TOOL_CHOICE": "true",
            "SM_VLLM_TOOL_CALL_PARSER": "hermes",
            "SAGEMAKER_ENABLE_LOAD_AWARE": "1",
        },
    },
)
print(f"Model created: {IC_MODEL_NAME}")

sagemaker_client.create_endpoint_config(
    EndpointConfigName=IC_ENDPOINT_CONFIG_NAME,
    ExecutionRoleArn=EXECUTION_ROLE,
    ProductionVariants=[
        {
            "VariantName": "variant1",
            "InstanceType": INSTANCE_TYPE,
            "InitialInstanceCount": 1,
        }
    ],
)
print(f"Endpoint configuration created: {IC_ENDPOINT_CONFIG_NAME}")

sagemaker_client.create_endpoint(
    EndpointName=IC_ENDPOINT_NAME,
    EndpointConfigName=IC_ENDPOINT_CONFIG_NAME,
)
print(f"Endpoint creation initiated: {IC_ENDPOINT_NAME}")

print("Waiting for endpoint to reach InService status (this takes 5-10 minutes)...")
waiter = sagemaker_client.get_waiter("endpoint_in_service")
waiter.wait(
    EndpointName=IC_ENDPOINT_NAME,
    WaiterConfig={"Delay": 30, "MaxAttempts": 40},
)
print(f"Endpoint is InService: {IC_ENDPOINT_NAME}")

sagemaker_client.create_inference_component(
    InferenceComponentName=IC_NAME,
    EndpointName=IC_ENDPOINT_NAME,
    VariantName="variant1",
    Specification={
        "ModelName": IC_MODEL_NAME,
        "ComputeResourceRequirements": {
            "MinMemoryRequiredInMb": 1024,
            "NumberOfCpuCoresRequired": 2,
            "NumberOfAcceleratorDevicesRequired": 1,
        },
    },
    RuntimeConfig={"CopyCount": 1},
)
print(f"Inference component creation initiated: {IC_NAME}")

print("Waiting for inference component to reach InService status...")
while True:
    desc = sagemaker_client.describe_inference_component(InferenceComponentName=IC_NAME)
    status = desc["InferenceComponentStatus"]
    if status == "InService":
        print(f"Inference component is InService: {IC_NAME}")
        break
    elif status == "Failed":
        raise RuntimeError(f"Inference component failed: {desc.get('FailureReason', 'unknown')}")
    time.sleep(30)

Ungakha izingxenye ezengeziwe ze-inference endaweni efanayo ukuze ubambe amamodeli amaningi anezilinganiso ezizimele kanye nokwabiwa kwezinsiza.

Cela izingxenye ze-inference

Ukuze ucele ingxenye ethile ye-inference, faka igama layo kumzila we-URL:

https://runtime.sagemaker..amazonaws.com/endpoints//inference-components//openai/v1

Isibonelo esilandelayo sibonisa izingxenye ezimbili ze-inference endaweni yokugcina okwabelwana ngayo, ngayinye eqondiswe iklayenti elihlukile le-OpenAI elabelana ngephuli yokuxhumana:

import httpx
from openai import OpenAI
from sagemaker.core.token_generator import generate_token

shared_http = httpx.Client()

client_a = OpenAI(
    base_url=(
        f"
        f"/endpoints/{IC_ENDPOINT_NAME}/inference-components/{IC_NAME}/openai/v1"
    ),
    api_key=generate_token(region=REGION),
    http_client=shared_http,
)

response = client_a.chat.completions.create(
    model="",
    messages=[{"role": "user", "content": "What is 42 * 3? Reply with the number."}],
)
print(f"Response: {response.choices[0].message.content}")
print(f"Connection pool active: shared_http is reusable across multiple IC clients")

Okwabelwana ngakho httpx.Client ivumela zombili izimo zeklayenti le-OpenAI ukuthi ziphinde zisebenzise izikhathi ze-TLS ezifanayo kanye nephuli yokuxhumana.

Hlanganisa nama-Strands Agents

I-Strands Agents ingumthombo ovulekile we-SDK yokwakha ama-agent e-AI. Ngenxa yokuthi ama-Strands Agents asekela abahlinzeki bemodeli ehambisana ne-OpenAI, manje usungasebenzisa ukugeleza komsebenzi kwama-ejenti amaningi ngokuphelele kungqalasizinda yakho ye-SageMaker AI. Lokhu kukunikeza ukuguquguquka kwezinhlelo zokusebenza ze-ejenti ngokulawulwa kwamaphoyinti okugcina azinikele. Idatha yakho ayilokothi ishiye i-akhawunti yakho, futhi ukhetha ngokuqondile ukuthi iyiphi imodeli esetshenziswa abenzeli bakho.

from openai import AsyncOpenAI
from strands import Agent, tool
from strands.models.openai import OpenAIModel
from sagemaker.core.token_generator import generate_token

@tool
def calculator(expression: str) -> str:
    """Evaluate a math expression."""
    return str(eval(expression))

strands_client = AsyncOpenAI(
    base_url=f",
    api_key=generate_token(region=REGION),
)

model = OpenAIModel(client=strands_client, model_id="", params={"temperature": 0.7})

coder = Agent(
    model=model,
    system_prompt=(
        "You are an expert Python developer. Write clean, well-documented "
        "Python code with type hints. Output ONLY the code, no explanation."
    ),
    tools=[calculator],
)

reviewer = Agent(
    model=model,
    system_prompt=(
        "You are a senior code reviewer. Review Python code for correctness, "
        "performance, and PEP 8 style. Give a concise review with specific suggestions."
    ),
    tools=[calculator],
)

Hlanza

Ukuze ugweme izindleko eziqhubekayo, susa amaphoyinti akho okugcina nezisetshenziswa ezihlotshaniswayo uma usuqedile. I-SageMaker AI endpoints ingenisa izindleko ngenkathi isesevisini, kungakhathaliseki ukuthi zithola ithrafikhi.

import boto3
sagemaker_client = boto3.client("sagemaker", region_name="us-west-2")

sagemaker_client.delete_inference_component(InferenceComponentName="")
sagemaker_client.delete_endpoint(EndpointName="")
sagemaker_client.delete_endpoint_config(EndpointConfigName="")
sagemaker_client.delete_model(ModelName="")

Isiphetho

Ngokusekelwa kwe-OpenAI-compatible API, i-Amazon SageMaker AI isusa umgoqo wokuhlanganisa phakathi kwalapho iningi lezinhlelo zokusebenza ze-AI likhona namuhla kanye nengqalasizinda ezidinga ukukala. Ungagcina ikhodi yakho ekhona, usebenzise noma yiluphi uhlaka oluhambisana ne-OpenAI, futhi usebenzise ukucatshangelwa kumaphoyinti okugcina azinikele nge-GPU, ukukala, nezilawuli zokuhlala zedatha ozidingayo. Ukuze uqalise, sebenzisa imodeli endaweni yokugcina yesikhathi sangempela ye-SageMaker AI usebenzisa isitsha esisekelwayo, faka i-SageMaker Python SDK, bese ukhomba iklayenti lakho le-OpenAI ekugcineni kwe-URL. Ukuze ufunde kabanzi, bona Sebenzisa i-SageMaker AI nge-OpenAI-compatible APIs ku- I-Amazon SageMaker AI Developer Guidenoma vula ikhonsoli ye-Amazon SageMaker AI ukuze udale isiphetho sakho sokuqala.

Mayelana nababhali