I-Ant Group's Robbyant Yembula i-LingBot-VA 2.0: Imodeli Yesenzo Sevidiyo Eyimbangela Eyakhelwe Ngokwemvelo Ye-Physical AI

URobbyant, iyunithi ye-AI efakwe ngaphakathi kwe-Ant Group, ukhiphe i- I-LingBot-VA 2.0.Imodeli yokuqala yesisekelo somdabu. Ichaza imodeli yesisekelo yesenzo sevidiyo sokukhohlisa irobhothi elijwayelekile. Ithimba locwaningo liqeqeshela kusengaphambili sonke isitaki ukuze sifanekiswe esikhundleni sokushuna kahle ijeneretha yevidiyo.

Iyini i-LingBot-VA 2.0?

Amamodeli amaningi esenzo sevidiyo aphinda asebenzise izingxenye ezimbili ezakhelwe ukwakhiwa kokuqukethwe kwedijithali. Enye i-VAE egxile ekwakhiweni kabusha. Enye iwumgogodla wokusabalalisa ividiyo oqondiswa kabili, enemojula yesenzo enamathiselwe.

Lokhu kudala imikhawulo emithathu. Ama-Pixel-reconstruction latent agcina ukubonakala kodwa aphethe ukwakheka komzimba okulinganiselwe. Ukuphindwaphindwa komsindo phezu kwamathokheni evidiyo kuhamba kancane kakhulu ekulawuleni iluphu evaliwe. Izinhloso zevidiyo ezijwayelekile azilokothi zifundise ukuthi izenzo ziwulolonga kanjani kabusha umhlaba.

Ukungafani kwesine kwesakhiwo. Ama-backbones asebenzisa ukunakwa kwe-bidirectional, kuyilapho ukulawula kwenzeka ngokuqhubekayo ngesikhathi. I-LingBot VA Version 1.0 ithuthukise leso sitaki saba yimodeli eyimbangela. Inguqulo engu-2.0 ilungiselela kusengaphambili i-DiT eyimbangela yomdabu.

Inguqulo 1: I-Semantic Visual-Action Tokenizer

Ngokwakhela kuleso sisusa, isigaba sokuqala singena esikhundleni se-VAE yokucindezelwa kuphela. Ukulandela i-RepWAM, i-tokenizer yengeza izinhloso ezimbili ekwakhiweni kabusha.

Ukuqondanisa kwe-Semantic kudonsela okufihlekile okubonakalayo ngakuthisha oqandisiwe we-Perception Encoder. Inhloso yesenzo esicashile ikhipha okuguquguqukayo okuhlangene phakathi kwamalatenti alandelanayo. Imodeli ye-dynamics ephambene ibikezela isenzo ngasinye esifihlekile. Imodeli ye-forward dynamics iwuhlukanisa ibe imephu yezokuthutha kanye nensalela.

Izifunda zomhlaba nezenzo manje zabelana ngesikhala esisodwa esicashile. Ngakho-ke ividiyo yewebhu engenamalebula inokuqondisa okuhlobene nesenzo.

Inguqulo yesi-2: I-Causal DiT Enokusakaza Kwevidiyo Ye-Sparse MoE

Phezu kwaleso sikhala, inguqulo yesi-2 iqeqesha i-DiT eyimbangela. Igcina ukwakheka kwe-Mixture-of-Transformers yenguqulo 1.0. Uchwepheshe wevidiyo kanye nochwepheshe besenzo babelana ngesizathu esisodwa sokuzinaka. Ngamunye unomzila ohlukile wokudlulisela phambili.

Zombili lezi zinhlobo zilinganisa i-asymmetrically. Ingcweti yevidiyo ithatha indawo ye-FFN yayo eminyene ngesendlalelo esinemizila ye-MoE. Leso sendlalelo siphethe ochwepheshe be-SwiGLU abayi-128, umzila we-top-8, uchwepheshe oyedwa okwabiwe. Ukulinganisa komthwalo kulandela isu Lokubhalansila elisizayo-lokungalahleki. Uchwepheshe wesenzo ugcina i-FFN eminyene endaweni efihliwe engu-768.

Umgogodla wevidiyo cishe unamapharamitha angu-13.0B, cishe u-1.9B osebenzayo. Ngochwepheshe besenzo namakhanda e-MCP, ukuqeqeshwa kuhlanganisa cishe amapharamitha angu-15.3B. Cishe i-2.5B yenza kusebenze ithokheni ngayinye lapho kucatshangwa khona. Ukuqeqeshwa kusebenzisa inhloso yokugeleza okulungisiwe nge-hybrid Muon plus AdamW optimizer.

Lapho Isibonakaliso Sokuqeqesha Sivela Khona

Ngale kwezakhiwo, izinjongo ezimbili zilolonga lokho okufundwa yimodeli.

I-Multi-chunk prediction (MCP) ilungisa ukugadwa kwe-myopic. Ukuphoqa kukathisha kulawula ingxenye elandelayo kuphela, ukuze imodeli inciphise ukulahlekelwa ngokukopisha ukubukeka. I-MCP inamathisela amamojula amathathu angasindi abikezela izingxenye ezintathu ezilandelayo. Ekukhishweni kwayo kufane nokunemba kwesisekelo sezinyathelo ezingu-45k ezinyathelweni ezingu-20k, isivinini sokuqeqeshwa esingu-2.3x.

Ngaleso sikhathi, izinjongo ezinhlanu ziqeqeshwa ngokuhlanganyela esikhundleni sokumiswa esiteji: i-T2I, i-T2V, i-TI2VA, i-ICL, nokuqeqeshwa ngokubambisana kwerobhothi labantu. Ukusampula kulandela uhlelo olumahhadla-kuya-ku-fine, kusukela ekubukekeni kuya ekulawuleni isenzo sevidiyo. Ukugcina yonke inhloso iphila kugwema ukukhohlwa ezangaphambili.

Ukuhlela kwe-Hierarchical

Ukulawula izinga le-Chunk akukwazi ukulandelana amagoli omkhathizwe omude. Ngaphezulu kwenqubomgomo ngakho-ke kuhlezi umhleli we-VLM, i-LoRA ecushwe ngombhoshongo oqandisiwe. Ikhipha i-JSON ehlelekile: yenziwe, isiyalo, i-generation_instruction, local_scene_description. Isebenza cishe ku-2 Hz ngemuva kwebhafa eyabiwe engavumelanisiwe. Inqubomgomo iyifunda emngceleni ngamunye we-chunk, ngakho ukubambezeleka komhleli akuvimbeli ukusebenza.

Ukubonisana kusengaphambili

Noma kunomgogodla omncane, ukusetshenziswa kuhlasela ibhodlela le-serial. Uma irobhothi lilindile, ukubambezeleka kwemodeli kuba ukubambezeleka kokulawula.

Ngakho-ke Ukubonisana kusengaphambili kusebenzisa ukubikezela nokwenza njengokusakazwa kwe-asynchronous. Ngenkathi irobhothi lenza i-chunk a_t, uchwepheshe wevidiyo ucabanga umphumela wawo. Uchwepheshe wesenzo unquma a_{t+1} kusukela kulokho.

Ukugijima phambili kuyingozi ukukhukhuleka. Ngakho-ke ukubuka ngakunye okubuyayo kubhalwa ngekhodi z_{t+1}, kusula okucatshangelwayo. Ukulahlekelwa kwesisekelo se-forward-dynamics kuqeqesha uchwepheshe wevidiyo wale ndima.

# Pseudocode for the asynchronous rollout (Sec. 2.3.7, Eq. 29).
# Not runnable: policy, executor and encode() are placeholders.

C = init_kv_cache(encode(obs_0))            # feedback-grounded cache C_t
a = policy.action_expert(C)                 # cold start: first action chunk a_0

while not done:
    executor.start(a)                       # execution stream, non-blocking

    C_tmp  = C + [a]                        # prediction stream: C_t u {a_t}
    z_hat  = policy.video_expert(C_tmp)     # forward dynamics -> imagined z_{t+1}
    a_next = policy.action_expert(C_tmp + [z_hat])

    obs = executor.wait()                   # real observation of a_t returns
    C   = overwrite(C_tmp, z_hat, encode(obs))   # re-ground: z_hat <- true z_{t+1}
    a   = a_next

Ukusebenza

Ngakho-ke, ukuhlolwa kuhlanganisa ukulingiswa kanye nehadiwe yangempela. Ku-RoboTwin 2.0, yonke imodeli iziqeqesha ngemibukiso engu-2,500 ehlanzekile kanye neyi-25,000 engahleliwe, kuyo yonke imisebenzi engu-50.

I-Distillation inciphisa isampula yevidiyo isuka ezinyathelweni ezi-5 iye kwezi-2, futhi isampula yesenzo sisuke ku-10 siye kwezi-2. Izinjini ze-FP8 TensorRT, inqolobane ye-KV ephejiwe/emagebhugebhu enokunakwa kwe-FlashInfer, kanye nokususwa kwekhanda lokusingatha ohlangothini kunikeza okusele.

# Reproduces Table 3 of the report exactly. Runnable as-is.
K = 32  # low-level control steps inside one generated chunk

stack = [("BF16 PyTorch async rollout baseline", 927),
         ("+ Consistency distillation",          466),
         ("+ Low-precision compiled execution",  369),
         ("+ Long-horizon attention optimization", 272),
         ("+ Runtime overhead reduction",        142)]

for name, ms in stack:
    print(f"{name:40s} {ms:4d} ms  {round(1000 / ms * K):4d} Hz")

print("end-to-end speedup:", round(927 / 142, 1), "x")

Inguqulo 1.0 vs Inguqulo 2.0

I-tokenizer ablation ihlukanisa umugqa wokuqala. Ukushintsha i-WAN2.2 VAE ye-semantic tokenizer kuphakamisa imodeli engu-1.3B isuka ku-78.0 iye ku-86.6.

Sebenzisa Amacala Nezibonelo

Ngalé kwamabhentshimakhi, izimo ezine zokusebenzisa ziyagqama.

• Ukugibela amashothi ambalwa: Umbiko uthi imodeli ijwayela kusuka emibonisweni eyi-10 kuye kweyi-15. Ukuhlola komhlaba wangempela kusebenzisa amademo asebenza ngocingo angama-20 ngomsebenzi ngamunye. Indawo yokuhlola yemisebenzi eminingi eminingi ihlanganisa yonke imisebenzi emine ehloliwe.
• Ukulawula okunesimo sokubonisa: Ukufunda ngaphakathi kokuqukethwe kuvumela ividiyo yokuboniswa komuntu ukuthi ingene esikhundleni somyalelo wombhalo. Ngemva kokulungiswa kwemisebenzi emine eboniwe, inqubomgomo yenze okuqanjiwe okungabonakali. Isibonelo esisodwa: “faka i-calabash epuletini eliluhlaza.”
• Ukukala idatha eshibhile: Isandla sokuqeqeshwa ngokubambisana kwerobhothi lomuntu sibeka endaweni yesenzo samarobhothi. Isandla ngasinye siba i-parallel gripper ebonakalayo. Ikhophasi ye-egocentric ihlanganisa iziqephu ezingu-65.4k.
• Ukulawula okusebenzayo: Imibukiso ihlanganisa i-Air Hockey kanye nebhande lokuthutha, lapho inqubomgomo ilindela izinto ezihambayo.

Okuthathwayo Okubalulekile

• Iqeqesha kusengaphambili i-DiT yesenzo sevidiyo eyimbangela kusukela ekuqaleni esikhundleni sokujwayela ijeneretha yevidiyo.
• Ithokheni ye-semantic ibeka izimo zomhlaba kanye nezenzo ezicashile endaweni eqondanisiwe eyodwa.
• Ukusakazwa kwevidiyo ye-Sparse ye-MoE: ~2.5B yamapharamitha angu-15.3B esebenza ngethokheni ngayinye.
• Ukubona kusengaphambili Ukubonisana kweqa isibikezelo nokwenziwa, okusekelwe kabusha kukho konke ukuqaphela kwangempela.
• I-Chunk latency 927 ms kuya ku-142 ms; ukulawula kwe-async 35 Hz kuya ku-225 Hz.

I-Interactive Dynamic Explainer

Hlola Iphepha futhi Ikhasi Lephrojekthi. Futhi, zizwe ukhululekile ukusilandela Twitter futhi ungakhohlwa ukujoyina wethu 150k+ML SubReddit futhi Bhalisela ku Iphephandaba lethu. Linda! ukutelegram? manje ungasijoyina kuthelegramu futhi.

Udinga ukusebenzisana nathi ekuthuthukiseni i-GitHub Repo yakho NOMA Ikhasi Lobuso Lokugona NOMA Ukukhishwa Komkhiqizo NOMA I-Webinar njll.? Xhumana nathi

_{Qaphela:Sibonga ithimba le-Ant Research ngobuholi bemicabango/Izinsiza zalesi sihloko. Ithimba le-Ant Research lisekele lokhu okuqukethwe/isihloko ukuze siphromothwe.}