Meta Fair issued by Code World Model (CWM): 32-Parameter Open-Weight Open-Weight LLM, Promote Research in Code and Earth General Studies

Meta Fair has been issued Code World Model (CWM)32 billed decoder-only graphic parader only in rubber Country model In the generation of the Code for training in the death of the killing and long-lasting partnership – not only static source text.

What's New: Learning Code by Execution?

CWM between trains in two major trajectories domestic families: (1) Pythons Translation Translorian translator that records local variables after each line done, and (2) Agentic partnerships within Dockorized repositories That is a photography planning, shell commands, and check the answer. This is why we are designed to teach the Semantics (the country from the world) than only syntax.

Rate collection, group located Images of impossible repository Since thousands of GitUB projects and trajectories with various measures with software engineering agent (“Forageragent”). Release Reports ~ 3m trajectories Over the opposite of ~ 10k images 3.15k regos, with a mutate correction and removal – a variety of repair.

Model and the context window

CWM is a Black, Decoder-only transformer (No MOE) with 64 layers, Gqa (48Q / 8kV), Swaglu, Rsnormbeside Ropped strings. Attention Alternates Local 8k including Worldwide 131k Sliving-Window blocks, enabling 131k tokens the successful context; Training uses a message mask-causal.

Training recipe (Pre → In the middle → Submit)

• General to escape: 8t tokens (Code-heavier) in the 8K context.
• Training in the middle: + 51k tokens, longest contexts (131k) in accordance with Python traces, foregeragent data, the Pred-taken Data, IR / Computers, Trillo Kernels, and Spirits.
• Training after: 100b-token sft For instruction + to express Multi-task rl (~ 172b-token) Alternatively ovarantical codes, statistics, and many SWAth-alged algorithm areas using the Grippo's Style and Small Tool (Bash / Edit / Provide / Submit).
• Liminated Limited Taking to Single 80 GB H100.

Benches

Research team shows the following Pass @ 1 / scores (To check the time to check on the application):

• SWE-Bench is confirmed: 65.8% (By measuring time).
• LiveCodebech-V5: 68.6%; LCB-V6: 63.5%.
• Matt-500: 96.6%; AIME-24: 76.0%; AIME-25: 68.2%.
• Cruxeval-Output: 94.3%.

The position of CWM research team as competitors with baselines of equal metal and large or closed models in SWE-Bench certified.

Similarly on the context in the verbal-bench task and metrics, see Butmark apps reports.

Why enlarge the world's examples of the code?

Release emphasizes two operating skills:

1. Prediction-signator: Official and monitoring of the tracking, CWM predicts stack frames (local people) and a step made each step in order as fixed reasons.
2. Agentic codes: Many consultation with tools for tools against real repos, verified with hidden tests and the identity of the patch. Setup training model to make local mistakes and produce endches of the end end (git diff) rather than snippets.

More information relevant to recognizing

• Tokenzer: LOLAMA-3 Family with control tokens stored; IDs kept used to resist track and compound partnerships during SFT.
• Making Attention: The 3: 1 Local: For the world Interleave is repeated across the depth; The training of the longest context has occurred The main sizes of Token Batch size to strengthen gradientents.
• The measurement of deception: Learning-rate-rate-rate-rate schedulations schedulations based on the depth of the internal law

Summary

CWM Step of PragMatic General code generation: Meta is a transforming transformer with a 32B variable in achieving the reading and Avention, an illegal assessment, issuance of unlimited codes without proof of production.

Look Paper, GitHub pagebeside The model in the kisses of face. Feel free to look our GITHUB page for tutorials, codes and letters of writing. Also, feel free to follow it Sane and don't forget to join ours 100K + ml subreddit Then sign up for Our newspaper.

Asphazzaq is a Markteach Media Inc. According to a View Business and Developer, Asifi is committed to integrating a good social intelligence. His latest attempt is launched by the launch of the chemistrylife plan for an intelligence, MarktechPost, a devastating intimate practice of a machine learning and deep learning issues that are clearly and easily understood. The platform is adhering to more than two million moon visits, indicating its popularity between the audience.