Welcome Le Chonk: Mistral Giant 4 on Hugging Face

Mistral Giant 4 (aka Le Chonk) is a multimodal, multilingual, Combination-of-Consultants (MoE) mannequin from Mistral. It takes textual content and pictures as enter and generates textual content. It has about 1T whole, 49B energetic parameter, and a context window of 1M tokens.

The mannequin is launched in two official variants:

1. FP8
2. NVFP4

There are additionally GGUFs created by Unsloth and the llama.cpp staff. It’s distributed below the Mistral Ecosystem Open-Weight License (an MIT spinoff that establishes limitations for model-as-a-service choices).

Mistral Giant 4 is especially sturdy for cybersecurity, coding brokers, and multimodality. On this put up, we have a look at how the mannequin works, its capabilities, structure and extra!

Desk of Contents

What’s new with Mistral Giant 4

Mistral Giant 4 combines a big pool of specialists with sparse consideration and a Pixtral imaginative and prescient encoder. The language mannequin selects a small group of specialists for every token, whereas its consideration indexer selects which earlier token positions to take care of. These are two totally different types of sparsity, the place one selects feed-forward networks, the opposite selects context positions.

You could find all of the checkpoints within the Mistral Giant 4 assortment.

Cybersecurity Capabilities

Mistral Giant 4 scores 82% on the Synthetic Evaluation Cyber Index check that asks a mannequin to breed an actual vulnerability in open-source software program after which patch the software program. Mistral experiences that a number of closed frontier fashions rating close to zero on the identical check as a result of they refuse to try the work. That is tremendous necessary for defenders, as a result of fixing a vulnerability often requires reproducing the vulnerability. In Mistral’s inside testing, Mistral Giant 4 was additionally helpful for malware evaluation, vulnerability prioritisation and writing detection guidelines, which the mannequin was not explicitly skilled for.

Mistral primarily positions the mannequin for defensive safety work. Vulnerability analysis and incident response usually appear as if assaults in mannequin enter, so closed supply filters can block them. A staff that is dependent upon an API may lose entry in the midst of an incident. Crucially, open weight fashions put each the utilization coverage and the deployment below the consumer’s management. That stated, the mannequin nonetheless declines dangerous requests: on cyber prompts from JailbreakBench, StrongREJECT and AgentHarm, its common refusal price is larger than another open weight mannequin.

For neighborhood builders, an open weight mannequin like Mistral Giant 4 unlocks a load of downstream use circumstances. You would construct your personal cyber system round your code base, crash logs, and malware samples. For instance, with TRL you would fine-tune smaller fashions from Mistral Giant 4 traces patching your code base. In truth, reproduce-then-patch duties use a verifiable reward, so it suits with RL environments that might be used to additional RL smaller safety fashions. Open weights additionally let researchers measure the mannequin’s refusal behaviour straight as a substitute of counting on the supplier’s description.

Briefly, cyber capabilities like ML4’s within the open are an enormous enhance for the neighborhood.

We have now interviewed Pierre Inventory concerning the cyber capabilities of the mannequin, you’ll be able to watch it right here.

Structure

Mistral Giant 4’s textual content decoder has 72 layers, a hidden dimension of 6,144, and 64 consideration heads. Its consideration blocks mix Multi-head Latent Consideration (MLA) with a DeepSeek Sparse Consideration (DSA) indexer (a path in different fashions corresponding to GLM 5.3). MLA retains a compressed KV illustration, and DSA selects as much as 2,048 key positions for every question.

For every decoder block, there’s a 3:1 RoPE to NoPE sample, i.e. 3 decoder layers make use of RoPE and each fourth layer makes use of NoPE. The mannequin additionally shares KV state throughout layers.
The primary decoder layer makes use of a dense feed-forward community. The opposite 71 layers use MoE blocks, by which a softmax router selects 4 of the 128 specialists for every token. One shared skilled at all times runs alongside the chosen specialists, for a complete of 5 energetic specialists per token.

Concerning RoPE, Mistral makes use of LeRope, a YaRN-scaled RoPE with one realized multiplier per frequency.

Imaginative and prescient encoder

Mistral Giant 4 employs a 1.6B Pixtral encoder. Pixtral has 30 layers, a hidden dimension of two,048, and 16 consideration heads. It makes use of 14 × 14 patches and a patch-merge projector with spatial merge dimension 2. You possibly can see the encoder pipeline beneath.

Benchmark Outcomes

Mistral Giant 4 delivers sturdy benchmark efficiency throughout coding, cybersecurity, multimodal understanding, and agentic workflows, positioning it as a aggressive addition to the open-weight ecosystem. As of October 29, 2026, the mannequin is the fifth greatest open weights mannequin on Synthetic Evaluation Intelligence index.

On coding, ML4 achieves 61.7% on DeepSWE v1.1, 59.4% on SWE-Atlas-QnA, and 28.3% on Terminal-Bench 4. Its mixed Coding Agent Index rating of 49.8% locations it forward of DeepSeek V4 Professional 0813 and Qwen3.8 Max.

Coding capabilities of ML4 prolong to information work, with 59.9% on AutomationBench and 1,393 Elo on AA-Briefcase, which consider enterprise workflows and long-horizon skilled duties.

Cybersecurity is one among Mistral Giant 4’s standout capabilities, with sturdy outcomes throughout real-world vulnerability evaluation and safety competitors duties. Three benchmarks spotlight the mannequin’s cyber efficiency:

• Synthetic Evaluation Cyber Index: ML4 ranks among the many prime 5 fashions globally on this impartial analysis of fashions’ potential to search out and repair software program vulnerabilities.
• Vulnerability replica and patching: It achieves 82%, the very best reported rating on this check inside the index. The duty requires reproducing an actual vulnerability in open-source software program after which patching it, testing each investigation and remediation.
• Cybench: ML4 solves 93% of a set of 40 challenges drawn from safety competitions, demonstrating sturdy efficiency on hands-on safety issues.

Mistral Giant 4 additionally demonstrates 93.3% assault resistance on Lakera’s B3 benchmark, including proof of robustness in opposition to prompt-injection assaults. Past benchmarks, inside testing confirmed promising ends in malware evaluation, vulnerability prioritization, and detection-rule era.

Inference Assist

Mistral Giant 4 is shipped with day-0 assist for transformers, vLLM and llama.cpp. GGUF quants can be found from GGUF and the llama.cpp staff. We offer choices to deploy remotely with Inference Endpoints, and use serverless inference by means of Inference Suppliers.

Transformers

The Transformers implementation offers AutoModelForMultimodalLM, with AutoProcessor getting ready the dialog and picture inputs. Mistral Giant 4 can take low, medium, excessive as reasoning effort. The default is excessive, and word that reasoning can’t be disabled, so utilizing reasoning_effort: none would nonetheless use excessive.

To run the mannequin, improve the transformers package deal

pip set up --upgrade transformers

import os
from transformers import AutoProcessor, AutoModelForMultimodalLM, DistributedConfig

distributed_config = DistributedConfig(
    tp_size=int(os.environ["WORLD_SIZE"]),
    ep_size=int(os.environ["WORLD_SIZE"]),
)

model_id = "mistralai/Mistral-Giant-4-1T-A52B"
processor = AutoProcessor.from_pretrained(model_id)
mannequin = AutoModelForMultimodalLM.from_pretrained(
    model_id,
    distributed_config=distributed_config,
)


text_messages = [
    {
        "role": "user",
        "content": "Who are you?"
    }
]


multimodal_messages = [
    {
        "role": "user",
        "content": [
            {
                "type": "image",
                "image": "https://huggingface.co/datasets/huggingface/cats-image/resolve/main/cats_image.jpeg",
            },
            {
                "type": "text",
                "text": "How many cats are in the image?",
            },
        ],
    },
]

for messages in [text_messages, multimodal_messages]:
    inputs = processor.apply_chat_template(
        messages,
        add_generation_prompt=True,
        tokenize=True,
        return_dict=True,
        return_tensors="pt",
        reasoning_effort="medium",
    ).to(mannequin.system).to(mannequin.dtype)
    input_len = inputs["input_ids"].form[-1]

    outputs = mannequin.generate(**inputs, max_new_tokens=100)
    if int(os.environ["RANK"]) == 0:
        print(processor.decode(outputs[0][input_len:], skip_special_tokens=False))

You possibly can run the script above by saving it in a python file script.py after which run

torchrun --nproc-per-node 8 script.py

vLLM

vLLM helps the high-quality NVFP4 Mistral Giant 4 checkpoint, which has a smaller reminiscence footprint and requires NVIDIA Blackwell GPUs. The mannequin additionally ships with a DSpark drafter for extra speed-ups with vLLM. Launch the server like this:

vllm serve mistralai/Mistral-Giant-4-1T-A52B-NVFP4 
  --config-format mistral 
  --load-format mistral 
  --tensor-parallel-size 8 
  --decode-context-parallel-size 8 
  --enable-expert-parallel 
  --dtype bfloat16 
  --max-model-len 131072 
  --max-num-seqs 16 
  --max-num-batched-tokens 2048 
  --gpu-memory-utilization 0.85 
  --enable-prefix-caching 
  --enforce-eager 
  --mm_processor_cache_gb 0.0

As soon as the server is up and operating, you’ll be able to run the next.

curl -s http://127.0.0.1:8000/v1/chat/completions -H 'Content material-Kind: software/json' 
  -d '{"messages":[{"role":"user","content":"What is 17*23?"}],"max_tokens":512}'

Llama.cpp

Llama.cpp and Llama App already assist Mistral Giant 4.

You possibly can set up Llama App with the next command:

curl -LsSf https://llama.app/set up.sh | sh

Then you can begin the server as follows, indicating the quant suffix after the mannequin ID:

llama serve -hf ggml-org/Mistral-Giant-4-1T-A52B-GGUF:Q2_K

Right here’s the repository with the quants derived from the FP8 checkpoint, and that is the NVFP4 one, transformed from the official NVFP4 repo.

As soon as the server has began, you’ll be able to head to localhost:9931 to speak utilizing the built-in WebUI. From the net UI you’ll be able to add photos for inference and alter the reasoning lever.
You can too question the server as follows. Be happy to take a look at the API reference for multimodality, device calls and extra.

curl http://localhost:8080/v1/chat/completions 
    -H "Content material-Kind: software/json" 
    -d '{
        "messages": [
            {"role": "system", "content": "You are a helpful assistant."},
            {"role": "user", "content": "Write a limerick about python exceptions"}
        ]
    }'

You possibly can run image-text inference as follows.

curl -s http://127.0.0.1:8080/v1/chat/completions 
    -H "Content material-Kind: software/json" 
    -H "Authorization: Bearer " 
    -d '{
        "mannequin": "muse-spark-open",
        "messages": [
            {
                "role": "user",
                "content": [
                    {
                        "type": "text",
                        "text": "What is in this image?"
                    },
                    {
                        "type": "image_url",
                        "image_url": {
                            "url": "data:image/png;base64,..."
                        }
                    }
                ]
            }
        ],
        "temperature": 0
    }'

The llama.cpp staff ships the next quants: IQ2_XXS, Q2_K, Q4_K_M, NVFP4, Q8_0, and BF16 for additional conversions. The two-bit quants run on a single Apple Silicon Extremely laptop with 512 GB of unified RAM. The others require distributed inference. You can too run on a number of NVIDIA or AMD playing cards.

Working Q2_K on a single M3 Extremely, we bought a prefill throughput of about 90 tokens per second, and a era pace of 15 tok/s. On 4 x RTX6000 Professional, we noticed the next outcomes:

• NVFP4: 32K context dimension, prefill 130-590 t/s, gen pace ~10 t/s
• IQ2_XXS: 800K context dimension. At quick context sizes, prefill is 500 tok/s and era runs at 30 tok/s. Nonetheless, utilizing lengthy contexts efficiency drops to 129 tok/s (prefill) and three tok/s (era).

Nice-tuning with TRL

We additionally ship day 0 assist of high quality tuning Mistral Giant 4 utilizing TRL. All it’s important to do is improve to the newest model of transformers, trl and peft.

pip set up --upgrade transformers trl peft

Within the following instance we practice the mannequin to carry out multilingual pondering. The high quality tuned mannequin would have the ability to cause within the language the system immediate asks for!

The script runs on the FP8 variant on a node of 8xB300s.

import os

from datasets import load_dataset
from peft import LoraConfig
from transformers import AutoModelForImageTextToText, DistributedConfig
from trl import SFTConfig, SFTTrainer

world_size = int(os.environ["WORLD_SIZE"])
model_id = "mistralai/Mistral-Giant-4-1T-A52B"
mannequin = AutoModelForImageTextToText.from_pretrained(
    model_id,
    distributed_config=DistributedConfig(fsdp_size=world_size, ep_size=world_size),
    dtype="auto",
)

dataset = load_dataset("HuggingFaceH4/Multilingual-Pondering", cut up="practice")

dataset = dataset.map(lambda x: {"chat_template_kwargs": {"reasoning_effort": "excessive"}})

coach = SFTTrainer(
    mannequin=mannequin,
    args=SFTConfig(
        packing=True,
        max_length=4096,
        assistant_only_loss=True,
        learning_rate=1e-4,
        lr_scheduler_type="cosine",
        warmup_steps=10,
    ),
    train_dataset=dataset,
    peft_config=LoraConfig(
        r=16,
        lora_alpha=32,
        target_modules=r".*language_model.*.(q_proj|kv_a_proj_with_mqa|kv_b_proj|o_proj|gate_proj|up_proj|down_proj)",
    ),
)
coach.practice()

Inference Endpoints

Mistral Giant 4 can also be obtainable on Inference Endpoints, with a ready-to-deploy recipe on 8 x NVIDIA H200 GPUs. The recipe runs vLLM with the FP8 weights cut up throughout all eight GPUs utilizing tensor and skilled parallelism, and makes use of the mannequin’s built-in DSpark draft head for sooner era by means of speculative decoding. It serves as much as 131K tokens of context and processes as much as 16 requests at a time, and we noticed the next outcomes:

• Single request: time to first token of about 0.6 seconds for a 1K-token immediate, and a era pace of ~65 tok/s, reaching 120–138 tok/s on chat prompts the place DSpark’s drafts are accepted.
• Lengthy prompts: 16K-token prompts, 4 at a time, are prefilled at about 4,300 tok/s in whole, with a time to first token of 8–9 seconds.
• Underneath load: about 480 tok/s of whole era throughout 1,000 ShareGPT conversations.

You possibly can deploy it with the hf CLI as follows:

hf endpoints catalog deploy --repo mistralai/Mistral-Giant-4-1T-A52B --name mistral-large-4

As soon as the endpoint is operating, you’ll be able to question its OpenAI-compatible API, utilizing the endpoint URL and your Hugging Face token. The reasoning_effort discipline controls how a lot the mannequin causes earlier than answering, and may be set to low, medium, or excessive (default).

curl "$(hf endpoints describe mistral-large-4 --json | jq -r .standing.url)/v1/chat/completions" 
    -H "Authorization: Bearer $(hf auth token 2>/dev/null)" 
    -H "Content material-Kind: software/json" 
    -d '{
        "mannequin": "mistralai/Mistral-Giant-4-1T-A52B",
        "messages": [
            {"role": "user", "content": "Write a limerick about python exceptions"}
        ],
        "reasoning_effort": "medium"
    }'

The reply comes again in selections[0].message.content material, and the reasoning in selections[0].message.reasoning. The endpoint additionally helps device calls, structured outputs, picture inputs, and streaming by means of the usual Chat Completions fields.

Dell Enterprise Hub

The simplest technique to rise up and operating together with your Dell AI Servers (NVIDIA H200, B200, GB200 or GB300) is by putting in the dell-ai CLI:

pip set up dell-ai && dell-ai login

And use it to run an optimized recipe for Mistral Giant 4 on e.g., a NVIDIA B300 node.

dell-ai fashions deploy --model-id mistralai/Mistral-Giant-4-1T-A52B-NVFP4 
    --engine docker 
    --platform-id xe9780-nvidia-b300 
    --gpus 4

You could find extra about this launch within the Dell Enterprise Hub (DEH).

Strive it out

Seize the weights from the Mistral Giant 4 assortment, run them regionally with the GGUFs, or deploy them with Inference Endpoints. We won’t wait to see what you construct with Le Chonk. Share your fine-tunes, quants and different artefacts on the Hub!