Running AI locally starts with the model, not with an impressive graphics card. Pick the model and the tasks you want it to handle first. For a new computer running larger local models alongside your everyday software, we recommend at least 64 GB of system RAM. That is buying advice, not a hard minimum: you can still try a small model for free on the laptop you already own.
This article lays out the specifications for businesses in Belgium and the Netherlands. It covers how much memory modern models need, which shared-memory computers are available, and what RAM and graphics cards cost today. It also explains when a hybrid approach beats forcing everything to run locally.
For small models, the computer you already have is often enough. The largest open models need 128 GB of RAM or more, with two tiers in between. The table below sums them up; the numbers come from the model table and from our own measurements later in this article.
| Tier | What you need | What you can run |
|---|---|---|
Entry | The computer you already have, with 16 GB of RAM | Small models up to about 9 billion parameters, such as llama3.2 and Qwen3.5-9B, through Ollama or LM Studio |
Mid-range | A PC with at least 64 GB of RAM and a graphics card with 16 to 32 GB of VRAM | Models with 20 to 35 billion parameters, such as gpt-oss:20b and Qwen3.8:27b, for coding and tool use |
Large | A computer with 128 GB of unified memory, such as a DGX Spark, a Mac Studio, or a Strix Halo mini PC | Models around 120 billion parameters, such as gpt-oss:120b and Qwen3.5:122b-a10b |
Heavy | 128 GB of RAM and a 32 GB graphics card with a runtime such as Strata or Project Maya, or 256 GB of memory | The largest open MoE models, such as Qwen3.8-Flash-Next, GLM-5.3-Flash and DeepSeek-V4.1-Flash |
A local AI computer runs a language model in your own environment (a local LLM) rather than on a model provider's servers. It could be a PC with a discrete graphics card, a compact shared-memory computer, or a Mac with unified memory.
With fully local processing, your prompts and files do not need to go to an external model provider. But local does not automatically guarantee privacy: an agent can still use an external API or web search. You therefore decide which connections are allowed. The Ollama documentation, for example, explains how to disable cloud features entirely. In our own setups, Strata and Project Maya listen only on the machine itself (127.0.0.1); if you open a runtime like that to your network, set an API key as well.
Local inference does not incur per-token charges from a model provider. Hardware, electricity, updates, and support still cost money. And more parameters do not automatically mean a better answer: model architecture, training, and the specific task matter too.
Memory decides what you can run: system RAM, VRAM, and the context you need set the limit together. Check the memory budget for the exact model variant in the table below, not a rule of thumb.
System RAM has been underestimated in recent years. The old 32 GB guideline leaves little headroom if you want to run a larger model, an agent, a browser, and document processing at the same time. Quantization, context length, and your offload approach together determine what you need.
With a discrete GPU, system RAM and video memory are separate. A PC with 64 GB of RAM and a graphics card with 16 GB of VRAM does not suddenly have 80 GB of fast GPU memory. If many model computations move to the CPU because the weights do not fit in VRAM, performance can drop considerably. Targeted offload of an n-gram lookup table is different: the whole table does not need to be processed continuously. To see where a model actually runs, use nvidia-smi for VRAM and free -g for system RAM, or check your runtime's dashboard. With Ollama, ollama ps shows it directly.
Quantization reduces model weights by storing numbers at lower precision. The rule of thumb of roughly half a GB per billion parameters at 4-bit applies only to those weights. Scales, non-quantized components, image processing, temporary buffers, and the context cache add to that. In the session below, ollama ps reports 18 GB of memory use with a context setting of 32,768 tokens. That is one specific setup, not a fixed ratio between download size and memory use. Even with a MoE model, you must store all experts, not just the active ones. And a large context window on a model page does not mean you can use it for free: more context requires more memory. That session is a memory test, not an agent setting; the FAQ below explains what an agent needs on top of that.
Also choose fast NVMe storage with room for several models: large model packages can now require hundreds of GB each. Also check that your RAM is stable. On our workstation, four memory modules at their 6,000 MT/s EXPO speed were not stable, and most of a model download from that period turned out to be corrupt. They now run at 5,600 MT/s, and we check every model file against the SHA-256 checksum published on Hugging Face.
With 16 GB, you can run small models up to about 9 billion parameters. Models with 20 to 35 billion parameters need 16 to 32 GB of fast memory, and the largest open MoE models need 128 GB or more.
The selection below combines widely used families from the Ollama library with recent open models. It is not a ranking of the smartest AI. Model names link to Hugging Face. Package sizes come from the listed Ollama variants. For Qwen3.8-Flash-Next, they come from the Atomic Dynamic builds, for GLM-5.3-Flash from the Unsloth GGUF files, and for DeepSeek-V4.1-Flash from the Project Maya README. For the two newest models, we used the Ollama model pages (checked September 15, 2026). The memory advice applies to those specific builds. Download size is not the same as memory use: offload keeps part of the model on the SSD, while context, image processing, and parallel tasks require extra memory. The Strata and Project Maya additions in the memory column are our own measurements on a single machine: Qwen on October 15, the Maya models on October 21 and 22, 2026.
| Model and variant | Parameters | Model package | Practical memory budget | What to consider it for |
|---|---|---|---|---|
3B, dense | 2.0 GB | about 6 GB with 32K context | Learning to use local AI on existing hardware | |
9B, dense | 6.6 GB | 12 to 16 GB | Text, document questions, and image understanding with a small model | |
12B, dense | 7.6 GB | 12 to 16 GB | Text, reasoning, and multimodal tasks | |
21B, 3.6B active | 14 GB | 16 GB documented minimum | Reasoning, programming, and tool use | |
27B, dense | 18 GB | 18 GB measured on a 32 GB GPU | Programming, document analysis, and image analysis | |
35B, 3B active | 23 GB | about 32 GB | Coding, working with repositories, and calling tools | |
117B, 5.1B active | 65 GB | one GPU with 80 GB is documented | Heavier reasoning and agent tasks | |
122B, 10B active | 81 GB | 128 GB system with about 96 GB for the model | More complex text, image, and agent tasks | |
125B/6B active, plus n-gram and MTP | 84.9–110.5 GB (Atomic Dynamic GGUF, excluding vision projector) | From 64 GB unified on a Mac (suitable build, n-gram on NVMe); 128 GB fully in memory. Strata: about 79 tok/s on 32 GB VRAM plus 128 GB RAM | Multimodal agents, coding, experimental | |
320B, 18B active (MoE) | 199.7 GB (UD-Q4_K_XL, dynamic 4-bit) | 256 GB recommended for this quant; smaller quants on 128 GB; with Project Maya, EXL3 3.05 bpw (125 GB) runs here on 32 GB VRAM plus 128 GB RAM, about 30 tok/s | Agentic coding and multimodal work | |
748B (MoE) | about 340 GB (Maya-S, experts in 2-3 bits, incl. Engram tables) | With Project Maya, Maya-S runs here on 32 GB VRAM plus 128 GB RAM, with part of the model on NVMe; about 25 tok/s; plan for 350 to 400 GB of free NVMe space | Hard problems you can wait for; a second opinion | |
30B, dense, multimodal | 18 GB | 24 GB VRAM with Meta's 4-bit quant | Always-on local agents with tool use | |
30B MoE, 3B active | 25 GB | about 32 GB | Fast agent tasks with 1M context | |
744B, 40B active (MoE) | 239 GB (UD-IQ2_M, dynamic 2-bit) | 256 GB system for this quant; 1-bit build fits from 223 GB | Z.ai's flagship model: long-running coding and agent work |
For Qwen3.8-Flash-Next, the package is larger than its label suggests. Alongside the 125B main model with 6B active parameters, the package includes an n-gram lookup table and an MTP component, bringing the total to about 180B parameters. But not everything needs to stay in fast memory permanently. The Qwen README explicitly describes the n-gram table as suitable for offload to host memory, with asynchronous prefetching. With suitable runtimes, that table can also stay on NVMe, with only the required portions read into memory.
The Atomic Chat guide documents a small Atomic Dynamic build on a Mac with 64 GB of unified memory, keeping the n-gram table on SSD. So 96 GB is not necessarily too little. This does require suitable quantization and a runtime with n-gram offload; it is not a promise that every Ollama or LM Studio package works this way.
Users on X report roughly 22 to 40 tokens per second during text generation with NVMe offload on different DGX Spark setups, for example in a measurement at 22 tokens per second and a measurement at 40 tokens per second. The performance loss is limited in those tests, but the measurements in Hugging Face discussion #11 also show that prompt processing can become slower. Hardware, quantization, context, and runtime make a difference. Fully in-memory operation is generally faster and simpler, especially for long prompts; with suitable quantization, 128 GB is a practical starting point.
On our own workstation at Voltti, Qwen3.8-Flash-Next now runs on 128 GB of system RAM, with an RTX 5090 (32 GB of VRAM) and a Ryzen 9 9950X3D. We use Strata, a runtime built specifically for this model family, with Unsloth's 5-bit UD-Q5_K_XL quantization. Strata keeps the experts a conversation uses most in VRAM, the next ones in system RAM, and the rest on the NVMe drive, and it moves them between those tiers while it runs. In our measurement on October 15, 2026, that setup reached about 79 tokens per second after calibration, with a 512K-token context window (Strata itself calls that extension experimental). A new measurement on October 21 and 22 came to 78.8 tokens per second. That is one machine, not a benchmark record; the full setup and test method are in our guide to Strata on 128 GB of RAM.
Before that, the same model ran here on 64 GB of system RAM: as a 3.05-bit-per-weight EXL3 quantization in TabbyAPI with ExLlamaV3 as the runtime, at about 50 tokens per second (see the earlier TabbyAPI guide). So 64 GB can work for this model, as long as the quantization and runtime fit the model and the memory. Free RAM turns out to be the real limit, though. A runtime like this claims almost all system RAM, until only a few GB are left. A second model, a game, or a browser with many tabs does not fit next to it, and switching models means stopping one runtime and starting the other.
GLM-5.3-Flash from Z.ai is even larger than Qwen3.8-Flash-Next and gpt-oss:120b, but can run entirely locally with enough memory. The multimodal model has an MIT license and a context window of about 1 million tokens. Unsloth's dynamic 4-bit variant, UD-Q4_K_XL, is 199.7 GB; for that variant, we recommend a computer with 256 GB of memory. Smaller dynamic quants can run on 128 GB, with lower precision and limited room for context. The 1-bit variant UD-IQ1_S, for example, is 93.1 GB. That is just the model package: the full context window does not automatically fit alongside the weights. Be careful with the Ollama model page: it only offers glm-5.3-flash:cloud, hosted by Ollama in the US and Europe, not on your computer. For local use, run Unsloth GGUFs through Unsloth Desktop or the appropriate llama.cpp build from the Unsloth guide.
We also run GLM-5.3-Flash on that same 128 GB workstation, through Project Maya, a fork of Strata built for this model. Maya reads the 3.05-bit-per-weight EXL3 quantization (125 GB) and keeps the experts that do not fit in VRAM or system RAM on the NVMe drive; its README marks that EXL3 route as experimental and NVIDIA-only. In our measurement on October 21 and 22, 2026, with Maya 1.1.21, it reached about 30 tokens per second with a 512K context: about 40 percent of the Qwen setup's speed, but for a much larger model (see our guide to Project Maya).
The 256 GB advice still applies if you want to keep UD-Q4_K_XL fully in memory, but it is no longer the only local route. Heavier still is GLM-5.3, Z.ai's flagship model (744B total, 40B active): Unsloth's dynamic 2-bit build is 239 GB and runs on a computer with 256 GB of memory.
The largest model on that same workstation is DeepSeek-V4.1-Flash, an open model from DeepSeek with 748 billion parameters and an MIT license. It does not fit in 128 GB of system RAM. Project Maya has a separate engine for it, with its own quantization: Maya-S, with the experts in 2 to 3 bits. With the Engram tables included, that comes to about 340 GB, so plan for 350 to 400 GB of free space on a fast NVMe drive. While it generates, the engine reads part of the model from that drive.
In our measurement on October 21 and 22, 2026 (Maya 1.1.21, 262K context), it reached about 25 tokens per second, with roughly 3 seconds of waiting for the first token. In practice, it takes over the whole machine: anything you start next to it pushes experts out of memory, and those then have to come from the SSD again. The installation and all measurements are in our guide to DeepSeek-V4.1-Flash with Project Maya.
Four large open models now run on one machine, but never at the same time: only one fits in memory, and switching means stopping one runtime and starting the other. That is why each model has its own role. Qwen3.8-Flash-Next is our default for code and agent work: it is the fastest, and it scored full marks in both rounds of nine tasks from our own work. GLM-5.3-Flash is the second opinion, at about 40 percent of that speed. DeepSeek-V4.1-Flash also scored full marks, but it writes much more slowly. We use it for hard problems we can wait for, not for interactive work. When the cloud model stopped in one of our sessions, Qwen3.8-Flash-Next took over the work. Our article on a local model as a fallback describes how that went.
The fourth model, Ornith-1.5-122B, runs through TabbyAPI with ExLlamaV3. It uses just over 60 GB of system RAM, so it leaves the most room free, but it scored the lowest on our tasks. Those tasks measure careful output, not intelligence. All the numbers are in our side-by-side comparison of the four models: one machine, one session, not a general benchmark.
Two newer models also focus specifically on local agents. Muse Glimmer from Meta (August 10, 2026, Apache 2.0) is a dense 30B model with image input and 128K context; Meta's own 17 GB 4-bit quant fits in 24 GB of VRAM, so a single consumer graphics card is enough. Nemotron 3.5 Lightning from NVIDIA (August 11, 2026) is a MoE model with 30B total and only 3B active parameters, with a context window of up to 1 million tokens. The Ollama package is 25 GB; NVIDIA lists a single DGX Spark or RTX 5090 as the minimum setup. For anyone who wants an agent to keep working continuously, these two lower the barrier considerably.
The right runtime depends on the size of the model. For small and mid-size models that fit in your VRAM or system RAM, start with Ollama or LM Studio, both free. Ollama runs a model with a single command; LM Studio offers a graphical environment for trying models side by side.
Ollama runs on NVIDIA cards, on many AMD Radeon cards through ROCm, and on Apple silicon through Metal. Before you buy, check whether your card is listed in the Ollama hardware documentation.
For large MoE models that barely fit, or do not fit, in VRAM plus system RAM, you use a runtime that spreads the experts across VRAM, RAM, and NVMe. Strata does that for Qwen3.8-Flash-Next and Project Maya for GLM-5.3-Flash and DeepSeek-V4.1-Flash; both runtimes are open source under the MIT license. If the model fits entirely in VRAM plus RAM, ExLlamaV3 with TabbyAPI is an alternative.
Expect more work on this route. Project Maya compiles an engine for your graphics card during installation, which its documentation says takes 10 to 30 minutes, and calibration on our own hardware took 10 to 11 minutes per runtime. While it runs, the machine is effectively dedicated to that one model. Our own setups run on Linux with an NVIDIA card.
Hermes needs at least 64,000 tokens of context for agent work, so check that limit in every runtime. Strata and Project Maya set it in their own configuration; ours is set to 512K tokens, or 262K for DeepSeek-V4.1-Flash. If you would rather not handle that yourself, Hermes Desktop installs and manages the local model server for you, and it picks a model build for each machine with at least that context window.
The terminal window below shows the Ollama starting route: a real historical session from our own test machine, with the original Dutch prompt and response preserved. We load Qwen3.8 in its 27B variant with 4-bit compression, ask one question, and then use ollama ps to check where the model was loaded.
$ ollama pull qwen3.8:27b-q4_K_M
$ ollama pull qwen3.8:27b-q4_K_M
pulling manifest
pulling ac3714bfddde: 100% ▕██████████████████▏ 931 MB
pulling f5f1dd8920d4: 100% ▕██████████████████▏ 16 GB
pulling 4c6a8e842ef0: 100% ▕██████████████████▏ 11 KB
pulling 448d29437397: 100% ▕██████████████████▏ 92 B
pulling 492b2922d38e: 100% ▕██████████████████▏ 215 B
verifying sha256 digest
writing manifest
success
$ ollama run qwen3.8:27b-q4_K_M
>>> Geef in exact één korte zin aan dat je lokaal draait.
Ik draai lokaal.
$ ollama ps
NAME ID SIZE PROCESSOR CONTEXT UNTIL
qwen3.8:27b-q4_K_M 25b843619e94 18 GB 100% GPU 32768 29 minutes from nowUnified memory lets the CPU and GPU share one memory pool, so a compact computer can load models that do not fit in a graphics card's VRAM. A discrete graphics card is no longer the only option.
Capacity is not speed, though: memory bandwidth, GPU compute, and software support still matter. A computer that barely fits the model is not automatically a comfortable work environment for a whole team.
Three families dominate this segment. The NVIDIA DGX Spark combines the GB10 Grace Blackwell Superchip with 128 GB of coherent LPDDR5x memory (273 GB/s) and a 4 TB NVMe SSD. NVIDIA describes the device as "Designed to build and run autonomous agents" and positions it for inference with "AI models up to 200 billion parameters." When we checked on September 7, 2026, the Founders Edition was in stock at Alternate in the Netherlands for € 6,399. Note that the system runs on Arm, so check compatibility with your other software.
The Mac Studio received a new generation on August 25, 2026, with M5 Max (up to 128 GB of unified memory) and M5 Ultra (up to 512 GB). According to Apple, the M5 Ultra lets you "run frontier-class models on device — no cloud tokens needed." At launch, the Apple Store in Belgium and the Netherlands listed starting prices of € 3,029 (M5 Max) and € 6,649 (M5 Ultra). Apple said at the time that deliveries would start on September 22 and that the 512 GB option would follow in late October. Those are base configurations, not maximum-spec AI configurations.
Then there is the AMD Strix Halo family: the Ryzen AI Max+ 395 combines 16 Zen 5 cores with Radeon 8060S graphics and supports up to 128 GB of unified memory. According to AMD, up to 96 GB can serve as graphics memory through Variable Graphics Memory. Compact mini PCs such as the GMKtec EVO-X2 with 128 GB cost around € 3,360. The LPDDR5x memory is soldered and cannot be upgraded later, so choose the capacity when buying. Above this segment is the DGX Station with 748 GB of memory (252 GB of HBM3e at the GPU plus 496 GB of LPDDR5X at the CPU). That is a different investment category, for which you request a quote based on your own workload.
What about running AI locally on a laptop? The same math applies, with less room. The GeForce RTX 5090 Laptop GPU, the top laptop card in the RTX 50 series, has 24 GB of VRAM, compared with 32 GB on the desktop card. A MacBook Pro with M5 Max can be configured with up to 128 GB of unified memory, shared by the CPU and GPU. For one person working with mid-size models, a laptop like that can be enough. For a team, one dedicated machine on the network still makes more sense: a laptop goes home with its owner and is switched off in the evening.
| Computer | Memory | Memory bandwidth | Price | Characteristics |
|---|---|---|---|---|
Standard PC + discrete GPU | 32-64 GB RAM + 16-24 GB VRAM | VRAM: 500-1,800 GB/s | € 2,000 - € 5,000 (with a 16 to 24 GB card; an RTX 5090 alone costs more) | Traditional option, fastest memory tier |
AMD Strix Halo mini PC | 128 GB unified (96 GB assignable to GPU) | about 256 GB/s | around € 3,400 | Plenty of memory, x86 software |
Apple Mac Studio M5 Ultra | up to 512 GB unified | 1.2 TB/s | from € 6,649 | Plenty of unified memory, MLX ecosystem |
NVIDIA DGX Spark | 128 GB coherent unified | 273 GB/s | around € 6,400 | Complete NVIDIA AI platform, Arm |
Enterprise server (multi-GPU) | hundreds of GB of HBM | several TB/s per GPU | Custom quote | Large models and multiple concurrent users |
RAM and graphics cards cost far more than a year ago, because AI data centers buy up a large share of the memory supply. Plan your budget on current quotes, not on launch prices.
TrendForce reported that conventional DRAM contract prices in the first quarter of 2026 were about 93 to 98 percent higher than in the previous quarter, and forecast another increase of 58 to 63 percent for the second quarter. Industry demand directs suppliers toward high-capacity server modules for AI servers, leaving less capacity for ordinary PC memory. Those are industrial contract figures, not the retail price of your DDR5 kit, but the Tom's Hardware RAM price tracker shows consumer modules following the same curve.
Graphics cards follow the same pattern. The TechSpot GPU pricing survey from late August 2026 found that card prices rose by about 15 percent in a single month across ten countries, with GeForce cards taking the biggest hits. The original suggested retail price is therefore no longer a reliable budget: compare complete quotes and include RAM, storage, the power supply, cooling, installation, and support. In this market, a unified-memory computer with fixed memory can sometimes cost less than an expandable PC that you need to upgrade immediately.
Typical retail price in Belgium and the Netherlands: the monthly median of all 2x16GB kits listed in Tweakers Pricewatch, including VAT. The dashed line shows the US price in dollars, excluding sales tax. The jump in late 2025 follows the shift in DRAM production toward memory for AI servers.
| Period | Belgium and the Netherlands (€) | United States ($) |
|---|---|---|
| Mar 2025 | €120 | $120 |
| Apr 2025 | €115 | – |
| May 2025 | €115 | – |
| Jun 2025 | €120 | – |
| Jul 2025 | €120 | $115 |
| Aug 2025 | €125 | – |
| Sep 2025 | €125 | $130 |
| Oct 2025 | €150 | – |
| Nov 2025 | €205 | – |
| Dec 2025 | €460 | $450 |
| Jan 2026 | €510 | – |
| Feb 2026 | €500 | – |
| Mar 2026 | €480 | $525 |
| Apr 2026 | €490 | – |
| May 2026 | €490 | – |
| Jun 2026 | €475 | $550 |
| Jul 2026 | €490 | – |
| Aug 2026 | €540 | – |
| Sep 2026 | €575 | $590 |
| Oct 2026 | €630 | $615 |
The GeForce RTX 5090 is a case in point. With 32 GB of video memory, this is the card from our own test session above. At launch in January 2025, the Founders Edition had a suggested retail price of € 2,369 in the Netherlands ($1,999 in the US). On October 12, 2026, the typical retail price in Belgium and the Netherlands was around € 6,750, almost three times as much. The average US market price was around $4,450 in October.
Typical retail price in Belgium and the Netherlands: the monthly median of all RTX 5090 models listed in Tweakers Pricewatch, including VAT. The dashed line shows the average US market price in dollars, excluding sales tax.
| Period | Belgium and the Netherlands (€) | United States ($) |
|---|---|---|
| Mar 2025 | €3,850 | $2,700 |
| Apr 2025 | €3,300 | – |
| May 2025 | €2,900 | – |
| Jun 2025 | €2,700 | – |
| Jul 2025 | €2,600 | $3,200 |
| Aug 2025 | €2,700 | – |
| Sep 2025 | €2,700 | $2,900 |
| Oct 2025 | €2,650 | – |
| Nov 2025 | €2,800 | – |
| Dec 2025 | €3,000 | $2,850 |
| Jan 2026 | €3,900 | – |
| Feb 2026 | €3,900 | – |
| Mar 2026 | €3,650 | $3,650 |
| Apr 2026 | €3,650 | – |
| May 2026 | €3,650 | – |
| Jun 2026 | €4,100 | $3,850 |
| Jul 2026 | €4,350 | – |
| Aug 2026 | €5,100 | – |
| Sep 2026 | €5,500 | $4,500 |
| Oct 2026 | €6,750 | $4,450 |
Five steps take a business from specs to running AI locally: map your hardware, match the model, install the software, connect the agent, and arrange management. The order is deliberate: hardware and model choice go together, and connecting tools and arranging management only make sense once something is running.
Each step also names what you can skip, because a pilot does not have to overhaul your entire IT environment. If your team operates in Belgium and the Netherlands, Voltti can guide these steps and handle the installation.
Start with an inventory of what is already there: which computers have how much RAM, is there a graphics card with VRAM, and how much NVMe storage is free? A compact model like Llama 3.2 3B runs on an ordinary work computer with 16 GB; heavier models follow the memory budgets in the table above. Do not buy anything new until you know what you already have: this first step costs time, not money.
Also decide where the AI computer will live. For a team, one central machine on the network is often more sensible than a powerful computer at every desk.
The table above combines download size and memory use per model variant. Use those numbers to choose: does the model you want fit in available memory, including context and the software already running? Afterward, check that the model runs where you expect it to: with ollama ps for Ollama, or nvidia-smi and free -g for other runtimes. A small model should sit entirely on the GPU; for a large MoE model, a split across VRAM, RAM, and NVMe is exactly the point. And for a team you do not need top-of-the-line quality: the model that handles your tasks well enough is the right choice.
Torn between a discrete graphics card and a unified-memory computer? The unified memory section above walks through that trade-off.
For small and mid-size local models, the simplest route is a runtime such as Ollama or LM Studio, followed by downloading your chosen model. The terminal session above shows exactly what that looks like. For large MoE models, choose a runtime that spreads the experts across VRAM, RAM, and NVMe, such as Strata or Project Maya, and allow extra time for installation and calibration. Install on a machine that is on when you need the agent: a work computer that shuts down every evening is not a reliable host for recurring tasks. Keep the installation deliberately small and document what you installed.
A model that only answers is not yet an agent. The next step is connecting the agent to the tools and files where your work already lives: email, your CRM, documents, or a chat channel the team uses anyway. You can read how we do this ourselves below, in the Linux computer section. Limit which accesses the agent gets from day one; the safety checklist for business data walks through that step by step.
A local setup is only finished when the maintenance is arranged: who updates model and software, who watches usage, and where do backups live? Encrypted backups go to a separate location in your own network, never to the machine itself. What happens when hardware fails and how to test a recovery is covered in our article on AI backup and recovery. Only once you have rehearsed a restore is the setup truly production-ready.
Curious how far existing hardware gets you? Our own Raspberry Pi build with Hermes and Home Assistant shows that a modest machine can already do plenty — as long as you match the tasks to it.
A dedicated Linux computer in your network makes sense once an agent handles recurring tasks. Processes stay available there, with defined permissions and human oversight for sensitive actions.
Running a model and putting an agent to work are two different things. The model generates answers; the agent connects those answers to tools, files, and follow-up actions. Our own Hermes installation at Voltti works this way, on a Linux machine we administer ourselves.
The agent computer does not have to be the same machine as the model server. A compact local node can coordinate tasks while a more powerful computer on the same network runs the model. Read more about that separation in when you need a Linux computer for an AI agent.
For most businesses, a combination is the most practical choice. The agent and task coordination stay in your own environment, and selected tasks go to an external model through an API. Which tasks those are depends on quality, speed, cost, and privacy.
Local models can now do more than short summaries: current families support coding, image understanding, and tool use. Some leading models are only available through an external service; more hardware does not make them runnable locally. Compare quality and speed on your own tasks before you invest in a more powerful workstation.
One caveat matters: text and files you include in an API request really do leave your network. So decide in advance what information may be sent, and have a person approve sensitive actions. Updates, recovery tests, and clear access permissions are also part of a usable setup. The full trade-off is in our comparison of local AI vs. cloud AI.
What does that look like in practice? Read about local and hybrid AI for businesses, about what stays local in hybrid AI, and about using AI securely and privately. Voltti installs and provides support for these setups in Belgium and the Netherlands, always with a custom quote.
Sharing this with Dutch-speaking colleagues or clients? There is also a Dutch version: AI lokaal draaien: welke computer heb je écht nodig?
Not to get started: small models also run on 16 GB. But for a new computer that needs to combine serious local AI, an agent, and your regular work software, 64 GB of system RAM is a sensible starting point. Think of it as headroom for now and for next year's models, not a hard system requirement. For the largest open MoE models, 128 GB is a more realistic starting point: on our own workstation, Qwen3.8-Flash-Next and GLM-5.3-Flash fill that memory to within a few GB.
Sometimes, but package size alone does not answer that question. If everything stays in memory, 96 GB of unified memory may accommodate a model package of roughly 80 GB, provided the runtime, context, and other software fit in the remaining space. It is not guaranteed. Qwen3.8-Flash-Next can also run on a Mac with 64 or 96 GB when using a suitable build and keeping the n-gram table on NVMe. That is a model-specific offload approach, not a general rule for all large models. For GLM-5.3-Flash, plan on 128 GB or more; the GLM section above lists the options. Always test the context length and tasks you actually want to use.
That depends on your goals. A discrete GPU offers the fastest memory for models that fit and is flexible to upgrade. Unified-memory computers such as the DGX Spark, Mac Studio, and Strix Halo mini PCs offer much more capacity for larger models in a compact, efficient form, but their memory is slower and cannot be upgraded. For most businesses, the choice is practical: which models do you want to run, and which software does your environment support?
Install Ollama or LM Studio on the computer you already own and try a small model. It costs nothing and immediately shows you what your current machine can handle. Only buy hardware once you know which models you really want to run. If you later want to run the largest MoE models, that usually takes more system RAM and a specialized runtime such as Strata or Project Maya.
Never on the AI computer itself. Encrypted backups go to a separate location in your own network, so a hardware failure or breach on the machine does not take your backup data with it. Also test now and then whether you can actually restore a backup: only then do you know it works.
In many cases, yes. Compact models up to roughly 9 billion parameters run on an ordinary work computer with 16 GB of RAM, which is why the first step of our installation path is an inventory of existing hardware. But a computer where the model barely fits is not automatically a good host for recurring tasks: if the machine shuts down every evening or is fully loaded with other work during office hours, you will soon want a dedicated machine on the network. The steps above help you make that call.
Yes. Small models also run on the CPU alone, using system RAM. Ollama shows this in ollama ps: 100% CPU means the model was loaded entirely into system memory. Expect slower answers than on a graphics card or a unified-memory computer. That is fine for trying a model or for short tasks, less so for an agent that works through long prompts all day. Start with a small model and check the speed on your own machine.
Not every model that runs locally can serve as the brain of an AI agent. An agent works through tools: it opens files, runs commands in the terminal, and uses a browser. A model without reliable tool calling can only chat. Hermes also needs at least 64,000 tokens of context for agentic work with tools. Ollama sets its default context window by VRAM: 4K tokens below 24 GB, 32K up to 48 GB, and 256K above that. Below 48 GB, raise it yourself; more context needs more memory. You can find more details in the Hermes guide to running Hermes with Ollama and the Hermes docs on local models. For heavy tasks, a cloud model can act as a fallback.