What a 4GB Model Is For:
NVIDIA Nemotron 3.5 Bolt E4B
NVIDIA has squeezed a tool-calling agent model into about 4GB of GPU memory. It will not replace a frontier model. It is built to be the worker that keeps the frontier model for the hard jobs.
By Serverless Ventures
Published: October 25, 2026 · Read time: ~7 minutes
The short version
On October 23, 2026, NVIDIA published Nemotron 3.5 Bolt E4B: Google's Gemma 4 E4B, retrained with reinforcement learning for instruction following, tool calling and conversation, then quantized to run in about 4GB of GPU memory.
On NVIDIA's own scorecard it beats Qwen 3.5 4B at following instructions, respecting system prompts, deciding whether to call a tool, and staying grounded in retrieved documents. It loses clearly on long, multi-step tool tasks and long-context retrieval.
That profile points to one job: the default worker in an agent stack. It takes the routine, high-volume calls on cheap local hardware and hands the hard ones to a bigger model.
What NVIDIA actually shipped
Bolt E4B is a post-training project, not a new architecture. NVIDIA took Gemma 4 E4B, which has about 8 billion parameters in total and about 4 billion effective, and ran reinforcement learning on it (RLVR and RLHF, using its NeMo-RL and NeMo-Gym tooling). The training data targeted tool use, function-calling schemas, instruction hierarchy and structured output. Every training sample was used in both reasoning and non-reasoning modes, at two non-reasoning samples for every reasoning one, so the model learns to be right without thinking out loud for long.
| Spec | Nemotron 3.5 Bolt E4B |
|---|---|
| Base model | Gemma 4 E4B (about 8B total, 4B effective) |
| Post-training | RLVR and RLHF for instruction following, multi-turn tool calling, conversation |
| Formats | Q4_K_M GGUF (llama.cpp, imatrix-calibrated) and BF16, each with a DFlash draft model for speculative decoding |
| Context | 64K tokens in NVIDIA's documentation |
| Inputs | Text and images in the Ollama package |
| Target envelope | One session at a time, 2.5K to 16K input tokens, under 500 output tokens |
| Intended uses | Voice and device assistants, interactive characters, embedded and industrial agents |
That target envelope says a lot. This is a model for short turns with a moderate amount of context: a voice assistant answering from a manual, a game character staying in role, a robot cell agent deciding which function to call next. NVIDIA is not pitching it for long essays or hour-long autonomous coding runs.
What "4GB" actually means
The 4GB is GPU memory while the model runs, measured on NVIDIA's target workloads. It is not the download size, and it does not mean any machine with 4GB of RAM will do. NVIDIA's own figures for the Q4_K_M build, one session at a time on llama.cpp:
| Device | Workload | VRAM | First token | Tokens/s |
|---|---|---|---|---|
| RTX 5070 (12GB) | 3K prompt | 3.5GB | 548 ms | 133 |
| RTX 5070 (12GB) | 32K cache | 4.3GB | 1,238 ms | 106 |
| RTX Spark (32GB) | 3K prompt | 3.1GB | 930 ms | 67 |
| RTX Spark (32GB) | 32K cache | 4.2GB | 1,988 ms | 52 |
| Jetson AGX Orin (32GB) | 3K prompt | 4.2GB | 3,687 ms | 24 |
| Jetson Orin Nano (8GB) | 3K prompt | 4.2GB | 5,785 ms | 15.5 |
Two caveats before you size hardware from this table. First, the Ollama e4b package is listed at 7.2 to 7.9GB to download, because it carries more than the bare quantized weights. Second, every figure is for one session. Run eight agents at once, or fill the context, and memory grows with each one's cache. Budget for your concurrency, not for the headline.
The speed column matters as much as the memory one. A consumer RTX card produces over 100 tokens a second, which is fast enough for voice. An 8GB Jetson board is around 15 a second with a six-second wait for the first token. That still works for a robot deciding its next action, but not for live conversation.
Where it wins, and where it does not
NVIDIA published results on 21 benchmarks against Qwen 3.5 4B and its own base model, Gemma 4 E4B. All three were run in BF16 with the same reasoning mode. A selection, grouped by the job each benchmark tests:
| Benchmark (what it tests) | Bolt E4B | Qwen 3.5 4B | Gemma 4 E4B |
|---|---|---|---|
| Following instructions | |||
| IFBench (instruction following) | 63.8 | 58.9 | 42.8 |
| IHEval (system prompt beats user) | 84.8 | 60.6 | 67.0 |
| StructEval-T (structured output) | 80.4 | 76.0 | 79.3 |
| MultiChallenge (multi-turn consistency) | 53.1 | 51.8 | 42.1 |
| Using tools | |||
| When2Call-MT (whether to call a tool) | 63.4 | 37.8 | 62.9 |
| RelyToolBench (reliable tool use) | 63.2 | 43.3 | 61.6 |
| BFCL-v3 (function calling) | 68.5 | 70.9 | 66.4 |
| ToolSandbox (stateful tool use) | 68.8 | 69.5 | 65.4 |
| Tau2-Bench (long multi-step tasks) | 55.1 | 75.1 | 38.5 |
| Context and grounding | |||
| GaRAGe (grounded retrieval) | 42.6 | 24.7 | 34.7 |
| LongMemEval (conversation recall) | 72.1 | 66.8 | 70.9 |
| LongBench v2 (long documents) | 50.5 | 59.9 | 49.3 |
| RULER 64K (long-context retrieval) | 87.0 | 95.1 | 86.5 |
Higher is better. Scores are on each benchmark's own scale, are reported by NVIDIA, and are for BF16 weights rather than the 4GB quantized build. Highest of the three is highlighted.
Read the table row by row and a clear profile emerges. Bolt E4B is disciplined: it does what the system prompt says, returns valid structure, holds back from calling a tool when it should not, and answers from the documents it was given instead of making things up. Qwen 3.5 4B has more stamina: it keeps a long chain of tool calls on track and finds facts buried deep in long inputs.
The Gemma column shows what NVIDIA's post-training added. Starting from the same base, IFBench rose from 42.8 to 63.8, MultiChallenge from 42.1 to 53.1, and Tau2-Bench from 38.5 to 55.1. Long-context scores barely moved. Reinforcement learning taught the model behavior. It did not give it a bigger working memory.
The worker slot
The practical way to use a model like this is not to pick it instead of a large model but to put it in front of one. Route every request to the small model first and escalate when the task is hard or its output fails a check.
- Classification, routing and triage
- Extraction into JSON for downstream systems
- Single tool calls and "should I call a tool at all" decisions
- Q&A over private documents that never leave the machine
- Voice and device assistants, in-role characters
- Short coding chores: explain, rename, write a test stub
- Long chains of dependent tool calls
- Reasoning over very long documents or codebases
- Complex coding, debugging and planning
- Anything where the small model's output fails validation
- Tasks where an error is expensive and the volume is low
Bolt's strengths suit this slot well. A router needs to follow its instructions exactly, produce output a program can parse, and know when to hand off. Those are the rows Bolt wins. The rows it loses are the ones you were going to escalate anyway.
The economics follow from the volume. In most agent systems the bulk of calls are small: classify this, extract that, call one function. If a 4GB model on a desktop GPU handles most of them well enough, cloud inference spend shrinks to the hard tail, latency on routine calls falls, and sensitive data stays on hardware you control. It is one more turn of the inference cost curve we traced in The Compound Startup Sandwich, and this time the cheap capability runs on hardware you already own.
Try it locally
The fastest path is Ollama, which runs it on Windows, Linux and Apple Silicon:
ollama run nemotron-3.5-bolt:e4b
For llama.cpp or your own serving stack, the weights are on Hugging Face as nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-4GB (GGUF) and nvidia/NVIDIA-Nemotron-3.5-Bolt-E4B-BF16, each with a matching DFlash draft model for faster decoding. NVIDIA's model page has the full benchmark and hardware tables. The Ollama package is released under the OpenMDW License 1.1. Read it before you ship anything commercial.
Test it before you trust it
Every number above comes from the vendor, measured on NVIDIA's chosen tasks and hardware. Before standardizing on Bolt E4B, Qwen 3.5 4B or anything else, run both small models side by side on 20 or more tasks taken from your real traffic, with a larger model as the quality baseline. Measure:
- Task completion rate: did it finish the job, judged against the larger model's answer?
- Tool-call accuracy: right function, right arguments, and no calls it should not make.
- Valid structured output: the share of responses that parse against your schema on the first try.
- Latency: time to first token and tokens per second on your hardware, not NVIDIA's.
- Memory under concurrency: VRAM with your real number of simultaneous sessions and context lengths.
- Escalation rate: how often the small model needs to hand off. This one number decides the economics.
A note on how we checked
This post began as an AI-assisted research thread, and it is a useful reminder of why vendor sources still matter. Three of that thread's claims did not survive a read of NVIDIA's own page. It said the benchmark table was measured on the quantized build, but NVIDIA states every result is BF16. It gave MultiChallenge as 55.6 against 52.2, where NVIDIA reports 53.1 against 51.8. And it treated the "4GB" as if it described the whole download, which Ollama lists at nearly twice that. None of these change the conclusion. All of them would change a hardware purchase. Check the primary source before you put a number in a plan.
Our read
NVIDIA did not pretrain this model. It took an open base and made it far better at agent behavior with reinforcement learning. Expect more "same base, specialized behavior" releases, and more teams doing the same for their own domains.
The rows Bolt wins (obey the system prompt, emit valid structure, know when not to act) are exactly the skills a front-line router needs. Judge small models on that job, not on frontier leaderboards.
A competent tool-calling model on an 8GB Jetson changes what can be built offline: kiosks, robots, vehicles, factory cells. The limit there is now latency, not whether the model is capable.
The caution: a vendor benchmark suite is chosen by the vendor, and several of the rows where Bolt leads by the widest margin (IHEval, When2Call-MT, and the role-play suite RoleBench at 67.0 against 53.7) match exactly what NVIDIA trained for. That is not cheating; it is what targeted post-training is supposed to do. But it means the published table shows where the model was aimed, not how it will handle your tasks. Your own evaluation is the only scorecard that counts.
Sources: NVIDIA Research, NVIDIA Nemotron 3.5 Bolt E4B 4GB, published October 23, 2026 (benchmarks, training method, hardware measurements); Ollama, nemotron-3.5-bolt (package sizes, inputs, license). Ollama lists a 128K context for the e4b tag; NVIDIA's documentation and benchmarks use 64K, which is the figure we use here.