Construct specialised multimodal fashions with LFM3

LFM3 is our new era of Liquid Basis Fashions (LFMs), constructed to be the place to begin for specialised intelligence. It brings textual content, imaginative and prescient, audio, omni, and choice fashions onto one shared spine, all within the 2B parameter vary. This implies you’ll be able to choose the mannequin in your modality, fine-tune it by yourself knowledge on a single GPU, and deploy it the place your software runs.

LFM3 brings 4 essential adjustments in comparison with our earlier LFM2 household:

• Extra functionality per parameter: a brand new tri-hybrid structure and coaching recipe enable LFM3-1.7B to outperform LFM2.5-2.6B whereas utilizing 35% fewer parameters.
• Each modality on one spine: six 2B-class fashions (textual content, imaginative and prescient, audio, omni, and choice) skilled from LFM3-1.7B-Base, together with our first omni mannequin and video enter for our imaginative and prescient mannequin.
• Constructed for the sting: the textual content mannequin matches in about 1 GB at 4-bit precision, new in-house imaginative and prescient and audio encoders velocity up processing, and the textual content mannequin’s 256K context window lets brokers run on-device.
• Made to be specialised: each mannequin fine-tunes on a single GPU, so you’ll be able to adapt it to your knowledge, terminology, and deployment atmosphere.

How we constructed a extra expressive spine

LFM2 confirmed {that a} hybrid of quick convolutions and a focus can run quick on-device with out sacrificing high quality. For LFM3, we used our hardware-in-the-loop structure search to revamp that structure for agentic use circumstances, which want extra functionality per parameter and quick inference over lengthy contexts. The search now covers 14 machine and backend mixtures, from telephones and a Raspberry Pi 5 to laptops, cloud CPUs, GPUs, and the Qualcomm Hexagon NPU.

LFM3-1.7B-Base has 24 layers and 4 essential adjustments:

1. Tri-hybrid layer combine: 13 gated quick convolutions, 5 sliding-window consideration (SWA) layers with a 256-token window, and 6 world consideration layers. The convolutions use a kernel measurement of seven, up from 3 in LFM2.
2. Head-wise gated consideration: a sigmoid gate scales every head’s output, so a head can swap off when it has nothing so as to add. This may help suppress activation outliers that harm quantization.
3. Partial RoPE: world consideration layers rotate solely the highest-frequency half of every head, so the opposite half matches on content material no matter distance. SWA layers preserve full RoPE.
4. Sandwich norm: every block normalizes each its enter and its output to maintain activations well-scaled.

The payoff reveals at lengthy context. Per parameter, LFM3-1.7B matches LFM2-1.2B on quick prompts throughout edge gadgets, and with 8K tokens in context it’s about 25% sooner. LFM3-1.7B-Base was pre-trained on 22T tokens with the identical 128K vocabulary as LFM2.5-8B-A1B. The ultimate 3.5T tokens are a mid-training stage on curated knowledge with a bigger share of agentic traces and gear calls, which additionally extends the context window to 256K tokens.

One spine for six fashions

Each LFM3 mannequin branches from LFM3-1.7B-Base. As a substitute of operating one massive general-purpose mannequin, you choose the one that matches your inputs, customise it, and run it on-device.

The imaginative and prescient and audio fashions additionally get two new in-house encoders:

• Imaginative and prescient encoder: it combines convolution and a focus and makes use of 3x much less compute than the SigLIP2 encoder in earlier LFM VL fashions. Our 164M-parameter encoder matches Qwen3.5-2B’s imaginative and prescient encoder in accuracy whereas requiring 2.2x fewer FLOPs (each measured with the identical LFM2.5-230M language spine). For video, it embeds every pair of sampled frames into one temporal slice, halving the temporal sequence size.
• Audio encoder and LiquidCodec: the audio encoder mixes sliding-window consideration and quick convolutions and is totally causal, so the mannequin can prefill whilst you converse, with time to first token as little as 50 ms. LiquidCodec, our new audio tokenizer, turns generated tokens into speech at about 87x actual time on a single CPU thread on Apple M4 Max and helps streaming decoding.

Find out how to use the LFM3 fashions

Every mannequin runs in transformers. The snippets under come from the mannequin playing cards, which additionally listing the opposite inference engines every mannequin helps.

LFM3-1.7B

Attain for LFM3-1.7B whenever you want native brokers, instrument calling, structured outputs, or a base for domain-specific fine-tuning. It’s post-trained to run in agentic harnesses comparable to Hermes Agent, Pi, and OpenClaw. It outperforms its predecessor, LFM2.5-2.6B, with 35% fewer parameters, and the most important positive factors are on agentic duties: on WebArena-Verified, it scores 38.42 in opposition to 3.02, and it even beats Qwen3.5-9B on lengthy duties in browsers and enterprise apps. Yow will discover extra particulars within the LFM3-1.7B mannequin card.

To run LFM3-1.7B, set up the most recent model of transformers:

pip set up -U transformers

Then load and run the mannequin:

from transformers import pipeline

generator = pipeline(
    "text-generation",
    mannequin="LiquidAI/LFM3-1.7B",
    trust_remote_code=True,
)

messages = [{"role": "user", "content": "What is C. elegans?"}]
output = generator(
    messages,
    do_sample=True,
    temperature=0.1,
    top_k=50,
    repetition_penalty=1.1,
    max_new_tokens=2048,
)
print(output[0]["generated_text"][-1]["content"])

For native agent setup, see the Agent Harnesses information.

LFM3-VL-2B

Attain for LFM3-VL-2B whenever you want on-device visible understanding. It’s the first LFM imaginative and prescient mannequin that takes video as enter; it causes step-by-step earlier than it solutions, and it finds objects and causes about how they’re positioned, a powerful basis for robotics. In contrast with LFM2.5-VL-3B, it has fewer parameters (1.9B in opposition to 3.1B) but raises the common throughout 62 imaginative and prescient benchmarks from 57.6 to 71.4, on par with Gemma 4 12B, a mannequin greater than six instances bigger. Yow will discover extra particulars within the LFM3-VL-2B mannequin card.

To run LFM3-VL-2B, set up transformers>=5.21.0 with its imaginative and prescient dependencies:

pip set up "transformers>=5.21.0" torch torchvision pillow

Then load and run the mannequin:

import torch
from transformers import AutoModelForImageTextToText, AutoProcessor

model_id = "LiquidAI/LFM3-VL-2B"

mannequin = AutoModelForImageTextToText.from_pretrained(model_id, dtype=torch.bfloat16, device_map="auto", trust_remote_code=True)
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)

messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://placecats.com/300/200"},
            {"type": "text", "text": "Describe this image."},
        ],
    }
]

inputs = processor.apply_chat_template(
    messages,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(mannequin.machine)

with torch.inference_mode():
    output_ids = mannequin.generate(
        **inputs,
        do_sample=True,
        temperature=0.1,
        top_k=50,
        repetition_penalty=1.1,
        max_new_tokens=16384,  
    )

generated_ids = output_ids[:, inputs["input_ids"].form[1] :]
print(processor.batch_decode(generated_ids, skip_special_tokens=True)[0])

See the LFM3-VL-2B mannequin card for instrument calling, instrument search, and on-device inference with llama.cpp.

LFM3-Audio-2B

Attain for LFM3-Audio-2B whenever you want voice chat, speech recognition, speech synthesis, or translation throughout 18 languages on a laptop computer or edge machine. It takes textual content or audio as enter and responds with textual content, speech, or each, and it might probably flip spoken requests into instrument calls. In contrast with LFM2.5-Audio-1.5B, it makes use of a brand new in-house audio encoder and speech codec, speaks with greater high quality (UTMOSv2 3.83 in opposition to 3.22), and provides audio understanding, general-purpose audio reasoning, and gear calling. Yow will discover extra particulars within the LFM3-Audio-2B mannequin card.

To run LFM3-Audio-2B, set up transformers>=5.20.2 with its audio dependencies:

pip set up "transformers>=5.20.2" torch speed up librosa soundfile

Load the mannequin and processor instantly from the Hugging Face Hub:

import torch
import soundfile as sf

from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "LiquidAI/LFM3-Audio-2B"
machine = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if machine == "cuda" else torch.float32

mannequin = AutoModelForMultimodalLM.from_pretrained(
    model_id, dtype=dtype, device_map=machine, trust_remote_code=True
).eval()
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
codec = processor.audio_tokenizer.to(machine=machine).eval()

Then ask a query aloud and get a reply in textual content and speech:

dialog = [
    {"role": "system", "content": "Respond with interleaved text and audio."},
    {"role": "user", "content": [{"type": "audio", "audio": "question.wav"}]},
    mixed_start,
]
inputs = processor.apply_chat_template(
    dialog,
    continue_final_message=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(machine)
with torch.no_grad():
    outputs = mannequin.generate(
        **inputs,
        max_new_tokens=1024,
        audio_temperature=1.0,
        audio_top_k=128,
        audio_repetition_penalty=1.1,
        audio_repetition_window=4096,
        return_dict_in_generate=True,
    )

    textual content = processor.decode(
        outputs.sequences[0, inputs["input_ids"].form[1]:],
        skip_special_tokens=True,
    )
    waveform = codec.decode(
        outputs.audio_codes[:1, :, :outputs.audio_lengths[0]]
    ).audio_values[0]  
print(textual content)
sf.write("reply.wav", waveform.float().cpu().numpy(), 24_000)

See the LFM3-Audio-2B mannequin card for reside speech recognition, speech synthesis, audio understanding, and gear calling.

LFM3-Omni-2B

Attain for LFM3-Omni-2B whenever you want one on-device mannequin that sees, listens, and talks again, rather than a separate ASR mannequin, vision-language mannequin, LLM, and TTS mannequin. It will probably reply spoken questions on a picture, watch and hearken to a video collectively, and switch what it sees and hears into instrument calls. It’s our first omni mannequin, with textual content, imaginative and prescient, and audio enter and each textual content and speech output, and it retains the core capabilities of the standalone LFM3 fashions, together with visible query answering, speech recognition, speech synthesis, and instruction following. Like LFM3-Audio-2B, it speaks via LiquidCodec, which turns generated tokens into speech sooner than actual time, so spoken replies stream because the mannequin generates them. On audio-visual video understanding, it beats fashions as much as 10.7B parameters, comparable to Qwen2.5-Omni-7B, on Every day-Omni, WorldSense, and AV-SpeakerBench. Yow will discover extra particulars within the LFM3-Omni-2B mannequin card.

To run LFM3-Omni-2B, set up transformers>=5.20.2 with its imaginative and prescient and audio dependencies:

pip set up "transformers>=5.20.2" torch torchvision torchcodec speed up pillow librosa soundfile

import torch
import soundfile as sf

from transformers import AutoModelForMultimodalLM, AutoProcessor

model_id = "LiquidAI/LFM3-Omni-2B"
machine = "cuda" if torch.cuda.is_available() else "cpu"
dtype = torch.bfloat16 if machine == "cuda" else torch.float32

mannequin = AutoModelForMultimodalLM.from_pretrained(
    model_id, dtype=dtype, device_map=machine, trust_remote_code=True
).eval()
processor = AutoProcessor.from_pretrained(model_id, trust_remote_code=True)
codec = processor.audio_tokenizer.to(machine=machine).eval()

Then ask a query a few picture by voice:

dialog = [
    {
        "role": "system",
        "content": "Answer questions about the provided image. Base every answer only on what can be seen. Respond with text.",
    },
    {
        "role": "user",
        "content": [
            {"type": "image", "image": "photo.jpg"},
            {"type": "audio", "audio": "question.wav"},
        ],
    },
]
inputs = processor.apply_chat_template(
    dialog,
    add_generation_prompt=True,
    tokenize=True,
    return_dict=True,
    return_tensors="pt",
).to(machine)

with torch.no_grad():
    outputs = mannequin.generate(**inputs, max_new_tokens=1024, generation_mode="textual content")

print(processor.decode(
    outputs.sequences[0, inputs["input_ids"].form[1]:outputs.sequence_lengths[0]],
    skip_special_tokens=True,
))

See the LFM3-Omni-2B mannequin card for speech output, video with its soundtrack, and gear calling.

d1-2B and d1-omni-2B

Attain for the d1 fashions whenever you want quick, structured choices as a substitute of generated textual content. Give them a state of affairs (a help ticket, an bill, an app display screen, a photograph, a voice word) and a listing of questions, they usually reply all of them in a single ahead go, every with a calibrated chance. Past sure/no, selections, and scores, they’ll now level at objects, learn values from paperwork, estimate numbers with a variety, and choose each possibility that applies. Yow will discover extra particulars within the mannequin playing cards for d1-2B and d1-omni-2B.

Between the 2 d1 fashions:

• d1-2B: select it for textual content, photos, and video. It replaces d1-3B with a 3rd fewer parameters and scores greater on all 4 of our choice suites, for instance, 75.5 in opposition to 64.0 on ImajevBench.
• d1-omni-2B: select it when your inputs embrace audio. If you happen to ask by voice as a substitute of in writing, it retains almost all of its accuracy.

To run d1-2B, set up transformers>=5.19 with its imaginative and prescient dependencies:

pip set up "transformers>=5.19" torch torchvision pillow

Then load d1-2B and ask it a number of questions on one help ticket in a single go:

import torch
from transformers import AutoModel

machine = "cuda" if torch.cuda.is_available() else "mps" if torch.backends.mps.is_available() else "cpu"
dtype = torch.float32 if machine == "cpu" else torch.bfloat16
mannequin = AutoModel.from_pretrained("LiquidAI/d1-2B", trust_remote_code=True, dtype=dtype).to(machine)


questions = {
    "refund": {"sort": "noul", "directions": "Is the shopper asking for a refund?"},  
    "staff": {"sort": "selection", "directions": "Which staff ought to deal with this?",
             "standards": {"billing": "Expenses, refunds, invoices", "transport": "Supply and parcels",
                          "technical": "App or website faults"}},
    "subjects": {"sort": "selection", "a number of": True, "directions": "Which subjects does the message elevate?",
               "standards": {"billing": "Funds and expenses", "supply": "Delivery and parcels",
                            "login": "Signing in to the account"}},
    "urgency": {"sort": "rating", "directions": "How pressing is that this ticket?", "standards": ["low", "medium", "high"]},
}
out = mannequin.system_one("I used to be charged twice and my parcel by no means arrived. Please refund one cost.", questions)
a = out["answers"]
urgency = a["urgency"]["legend"][str(int(a["urgency"]["score"]))]
print(spherical(a["refund"]["noul"], 2), a["team"]["choice"], a["topics"]["choices"], urgency)

For brevity, we solely embrace the instance for d1-2B. See the d1-omni-2B mannequin card for audio clips and spoken questions.

Advantageous-tune LFM3 in your activity

LFM3 is constructed to be specialised. Frontier fashions are highly effective generalists, however they don’t seem to be essentially specialists in your use case. LFM3 packs additional capability right into a small, quick mannequin, so a brief fine-tune can flip it right into a dependable professional in your activity, together with your knowledge, terminology, and instruments. As a result of it’s small, you’ll be able to deploy the fine-tuned mannequin the place your software runs, from gadgets to personal infrastructure to the cloud.

Each mannequin may be fine-tuned on a single GPU, and these notebooks stroll you thru an entire activity for the textual content, imaginative and prescient, audio, and omni fashions, from knowledge preparation to analysis.

Instance: educating LFM3 to behave

We adapt LFM3-VL-2B into three complementary robotics fashions for orchestration, navigation, and manipulation. The orchestrator operates a robotic arm via instrument calls: it grounds objects by pointing, measures them with depth instruments, invokes movement planners, and checks outcomes in subsequent digicam pictures. The navigator follows route descriptions utilizing current RGB observations, predicting spatial targets and a waypoint trajectory in simply 5 tokens. The manipulation VLA maps pictures, language, and robotic state to eight-step chunks of steady actions in a single ahead go. Collectively skilled throughout 15 embodiments, the VLA achieves 96.2% common success throughout the 4 LIBERO suites and 45.7% on RoboCasa-GR1 tabletop duties, producing motion chunks in below 50 ms on a Jetson AGX Orin with TensorRT FP16. These outcomes spotlight LFM3-VL-2B as a compact basis for robotics, spanning high-level planning to low-level management with real-time motion era on embedded {hardware}.

Watch LFM3-VL-2B deal with notion, navigation, and manipulation on this demo video.

Get Began with LFM3

All LFM3 fashions are open-weight and accessible on Hugging Face in the present day:

We won’t wait to see what you construct.

Quotation

If you happen to use this work, please cite the discharge weblog:

@article{liquidAI2026LFM3,
  writer = {Liquid AI},
  title = {Introducing LFM3: Each Modality for Specialised Intelligence},
  journal = {Liquid AI Weblog},
  yr = {2026},
  word = {www.liquid.ai/weblog/lfm3},
}