Skip to content
Aventail™ investigation into UK government AI dependence featured in Computer Weekly.Read the story
Aventail™
All guides

Local AI for a development team: three setups, sizing and rollout

12 min read

How to give a whole development team local AI for coding: a model on every laptop, one shared server, or a routed mix. Costs, failure modes and sizing numbers.

There are three ways to give a whole development team local AI for coding: put a model on every laptop, run one shared inference server, or route between the two with a rule that decides where each request runs. For more than a few developers, put the large model on a shared server, keep a small one on each laptop, and write the rule that chooses between them.

The table compares the three. The numbers under it are published measurements, read on 15 November 2026.

A model on every laptop

One shared inference server

A routed mix

Hardware cost

A machine with 32 GB or more for each developer, for the 27B to 30B coding models. Cost grows with headcount

One GPU server, and laptops of any size. As a reference, CloudRift rents an RTX 5090 for $0.65 an hour and an RTX PRO 6000 for $1.20

The laptops you have, one server, and provider usage on your own account if a rule allows cloud

Who maintains it

Every developer, unless a fleet tool pushes the model and settings

Whoever owns the server: runtime, model, access control and uptime

An admin who owns the routing policy, and the server owner

Where code goes

Nowhere. Prompts stay on the laptop

To the server, over your network. It stays inside if the server is yours and its port is not exposed

Where the rule says: the laptop, the server, or an approved provider

Failure behaviour

One laptop fails and one developer stops. Works offline once the model is installed

The server fails and everyone stops

The rule's fallback order applies. With no permitted destination the request fails with an error

Largest model

Set by the weakest laptop

Set by the server's GPU memory

Small on laptops, large on the server

Speed for one developer

23.7 tokens a second for a 30B model on an Apple M4 Max

74.9 tokens a second for the same model on an RTX 5090, shared with whoever else is asking

Depends on the destination chosen

1. A model on every laptop

Each developer runs a model with a local runtime such as Ollama or LM Studio and points the coding tool at localhost. Setup for one machine is in the guides for Claude Code with Ollama and OpenCode with Ollama.

It suits a small team where everyone has a recent machine with 32 GB or more. It is the only setup that works on a train. Three things go wrong as the team grows:

  • Uneven hardware. The developer with 16 GB runs a 9B model while a colleague runs a 27B model, and they get different results on the same task. The best local LLMs for coding lists what fits each memory size.

  • No shared configuration. Model, context length and endpoint live in each person's shell profile. Ollama starts with a 4K context below 24 GiB of VRAM, and coding agents need 64K, so one missed setting breaks tool calls.

  • No record. Nobody can show which machine sent which request where.

2. One shared inference server

One machine with a large GPU serves the model, and every coding tool points at it. Laptops need no extra memory. The team gets one model and one place to update it.

Three points decide whether it works:

  • The runtime's concurrency. Ollama processes one request at a time for each model by default. Raising OLLAMA_NUM_PARALLEL multiplies the context memory: four parallel requests at 64K need the memory of a 256K context. When the queue of 512 requests is full, Ollama returns a 503. The VMware test in the sizing section used vLLM, which batches concurrent requests; the trade-offs are in vLLM vs Ollama.

  • Access to the port. Ollama binds to 127.0.0.1 by default, and you change that with OLLAMA_HOST. Its local API accepts any key: the documentation marks the token as "required but ignored". Once the port is on the network, anyone who can reach it can use it, so put it behind your own access control and never on the public internet.

  • The single point of failure. A driver update or a stuck process stops the whole team. Decide who is on call for it.

3. A routed mix

Each laptop runs a small model for quick edits and for work offline. The server runs the large model. A rule on each laptop decides which one a request uses, and what happens when the first choice is unavailable. If policy allows it, the last fallback is a cloud provider on the company's own account.

This is the setup that needs a routing layer in addition to a runtime. Without one, each developer switches base URLs by hand and the "rule" is a wiki page. The questions to settle are the same whichever tool you use:

  • Which work may never leave the laptop or the server.

  • Which destination is first, and which fallbacks are allowed, in order.

  • What the developer sees when nothing is allowed. It should be an error, with no quiet switch to another provider.

  • How you check where a request ran.

Sizing: how many developers one server can carry

No published figure tells you the number for your team. The measurements below give the range.

What was measured

Hardware

Model and runtime

Result

Source

One stream

RTX 5090

Muse Glimmer-30B, 4-bit, llama.cpp

74.9 tokens a second; 233.4 with speculative decoding

Meta model card, August 2026

One stream

Apple M5 Max and M4 Max

Muse Glimmer-30B, 4-bit, ExecuTorch

26.6 and 23.7 tokens a second; 50.2 and 37.8 with speculative decoding

Meta model card, August 2026

One stream, small model

RTX 4070, 12 GB

Gemma 4 E2B Q4_K_M on Aventail™ Link

196 tokens a second (Ollama: 166)

Aventail™ benchmarks, June 2026

One stream, small model

Apple M2, 16 GB

Gemma 4 E2B Q4_K_M on Aventail™ Link

39 tokens a second (Ollama: 41)

Aventail™ benchmarks, June 2026

Total throughput, many requests

One RTX 4090, RTX 5090, RTX PRO 6000

Qwen3-Coder-30B AWQ on vLLM, 8K context

2,259, 4,570 and 8,425 tokens a second

CloudRift GPU benchmarks

Total throughput, many requests

Four RTX 5090

Qwen3-Coder-30B AWQ on vLLM, 8K context

12,744 tokens a second

CloudRift GPU benchmarks

Concurrent users inside a latency target

Four H100

gpt-oss-120b on vLLM, internal coding assistant

300 users

VMware Cloud Foundation blog, 30 April 2026

Concurrent users inside a latency target

Two H200

gpt-oss-120b on vLLM, internal coding assistant

85 users

VMware Cloud Foundation blog, 30 April 2026

How to use these numbers:

  1. Count concurrent sessions instead of headcount. VMware's figures are users active at the same moment, with pauses between turns. That works out at about 75 users per H100 and about 42 per H200 for a 120B model in their test.

  2. Expect less than the batch benchmarks. CloudRift measured an 8K context. VMware's developers sent far more: 70% of users grew from 5,000 to 50,000 tokens, 20% from 15,000 to 120,000, and 10% started near 75,000. Long prompts were what limited the server, through attention compute and memory bandwidth, and GPU memory was not the limit.

  3. Do not scale by GPU count. Four H100 carried about 3.5 times the users of two H200 in the same test.

  4. Start from the single-stream speed. In Meta's test, one stream on an RTX 5090 ran about three times as fast as on an M4 Max laptop with the same 30B model. Batching on a server raises the total, and each person's share falls as sessions overlap.

  5. Budget memory for parallel contexts. Weights are loaded once. Context is held for each parallel request.

  6. Measure with your own prompts. VMware's authors say it directly: "We cannot claim to know the optimal number of users for your deployment".

A workable first step is one server with a GPU that holds your chosen model, a short pilot with a few developers, and a count of how often requests queue.

Rollout steps with Aventail™

Aventail™ covers the routed mix. Aventail™ Link is the inference engine and gateway that runs on each laptop and on the server. Aventail™ Control holds the fleet configuration and the routing rules; Link evaluates the rules on the device, and hosted Control receives device and routing events, not prompts or responses. Aventail™ Connect points Claude Code, OpenCode and Codex at Link. Fleet configuration and routing policies are part of Control for Enterprise.

These steps follow Aventail™'s deployment guide and documentation. Include a Linux and a Windows machine in your pilot, so every platform your team uses is covered.

  1. Pick one workflow and its owner. Write down the data it handles and the destinations it may use.

  2. Create a fleet key. In Control, open Fleet operations, choose Add devices, then Create a key. The key is shown once. You need the Fleet Admin or Super Admin role.

  3. Enrol one test laptop. On macOS or Linux:

    curl -fsSL https://get.aventail.co.uk/install.sh | AVENTAIL_FLEET_KEY='YOUR_FLEET_KEY' sh

    On Windows PowerShell:

    $env:AVENTAIL_FLEET_KEY = 'YOUR_FLEET_KEY'
    try { & ([scriptblock]::Create((irm https://get.aventail.co.uk/install.ps1))) }
    finally { Remove-Item Env:AVENTAIL_FLEET_KEY }

    Confirm the device shows online in Devices. Packages exist for macOS on Apple silicon, Linux x64 and Windows x64.

  4. Deploy and serve a coding model. Link runs GGUF language models. Start serving, then send a harmless prompt. A downloaded file is not a running endpoint.

  5. Connect the coding tool. In Link, open Connected apps and choose Connect beside Claude, OpenCode or Codex. Claude Code needs at least 64K context, and Link warns below 128K. aventail connect --list shows the state from a terminal.

  6. Add the shared server. Enrol it the same way, serve the larger model, and turn on Serve other devices. Review its listener, authentication, TLS and firewall rules before other machines reach it: the docs note that the tested endpoint used HTTP and that Control sign-in does not protect the serving port.

  7. Write the routing policy. Tag the test devices and create a policy for the tag. A rule has a condition, a primary destination and up to three fallback hops, and the first matching enabled rule wins. Destinations are On device, a network peer, or a provider connection on your own account. A peer must offer the exact model named in the rule, and a peer can be a primary destination only, not a fallback hop. Peer and hosted routing are in beta.

  8. Check delivery, then a real request. Wait for the device's Delivery state to show applied. Send a harmless request and read the destination in Recent usage. Then stop local serving on a test device and confirm the result is the fallback you configured, or an error.

  9. Publish the configuration and roll out in rings. A fleet configuration revision lists the models and whether devices serve them. Apply it ring by ring, and check the applied revision and a response before you advance. Stopping a rollout does not reverse changes already applied, so record the previous configuration and who can restore it.

Each request runs on one destination. Link does not pool machines or split a request across them, so size the server on its own. Aventail™'s Free plan covers three connected devices. Starter is £50 a month excluding VAT with ten devices, and £5 a month for each extra device. The staged plan for laptops and servers together is in deploying a private AI workspace.

Which setup to choose

Your situation

Start with

A small team, all on 32 GB or more

A model on every laptop

Mixed laptops, code must stay on your network

One shared server

Travel, offline work or a server that must not stop the team

A routed mix with a small model on each laptop

Some work needs a frontier model

A routed mix with an approved provider as the last fallback

Frequently asked questions

How can I give my whole team local AI for coding?
Run one shared inference server with a 27B to 30B coding model, point each developer's coding tool at it, and keep a small model on each laptop as the fallback. Use a routing rule so the fallback is automatic and recorded.

How many developers can one GPU serve?
It depends on the model, the context length and how many sessions overlap. VMware reported 300 concurrent users on four H100 and 85 on two H200 for a 120B model in April 2026. Pilot with your own prompts before you buy.

Do we need a GPU server, or are laptops enough?
Laptops are enough if every developer has 32 GB or more and you accept that each person maintains their own setup. A server is the better choice when laptops vary or when you want one model for everyone.

Can a team share one Ollama server?
Yes. Set OLLAMA_HOST so it listens on the network and raise OLLAMA_NUM_PARALLEL above its default of 1. Ollama's local API does not check keys, so restrict who can reach the port.

Does our code leave the building with a shared server?
Not if the server is on your network and the tools point only at it. Check the coding tool's own telemetry and web tools separately, and check any fallback to a cloud provider.

What happens when the shared server is down?
Without a fallback, everyone stops. With a routing rule, requests go to the next permitted destination in order, and fail with an error if none is permitted.

Which model should a team standardise on?
One that supports tool calling and fits the server with room for parallel contexts. The current options by memory size are in best local LLMs for coding.

Sources

  1. Meta, Muse Glimmer-30B model card (speed table): https://huggingface.co/meta-models/Muse-Glimmer-30B

  2. CloudRift, GPU benchmarks for LLM inference: https://www.cloudrift.ai/gpu-benchmarks

  3. VMware Cloud Foundation blog, How many users can your LLM server really handle? (30 April 2026): https://blogs.vmware.com/cloud-foundation/2026/04/30/how-many-users-can-your-llm-server-really-handle/

  4. Aventail™, Benchmarks: Aventail™ Link vs Ollama (June 2026): https://aventail.co.uk/benchmarks

  5. Ollama, FAQ (concurrent requests, queue, network binding): https://docs.ollama.com/faq

  6. Ollama, Context length: https://docs.ollama.com/context-length

  7. Ollama, Anthropic compatibility: https://docs.ollama.com/api/anthropic-compatibility

  8. Ollama, Claude Code and OpenCode integrations: https://docs.ollama.com/integrations/claude-code and https://docs.ollama.com/integrations/opencode

  9. Aventail™, Deploying a private AI workspace across laptops and servers: https://aventail.co.uk/guides/private-ai-workspace-deployment

  10. Aventail™, Add devices: https://docs.aventail.co.uk/docs/fleet/overview

  11. Aventail™, Fleet configuration: https://docs.aventail.co.uk/docs/fleet/config-and-rollouts

  12. Aventail™, Routing: https://docs.aventail.co.uk/docs/fleet/routing

  13. Aventail™, Connect overview: https://docs.aventail.co.uk/docs/connect/overview

  14. Aventail™, Security and access: https://docs.aventail.co.uk/docs/guides/security

  15. Aventail™, Plan a deployment: https://docs.aventail.co.uk/docs/guides/deployment

  16. Aventail™, Compatibility and versions: https://docs.aventail.co.uk/docs/introduction/versions-and-supported-models

  17. Aventail™, Pricing: https://aventail.co.uk/#pricing

All sources were read on 15 November 2026.

We use optional cookies and tracking to identify visiting companies, load our contact form, measure Google Ads download clicks and understand page use with PostHog on our homepage, Connect and How it works pages, Ollama and LM Studio comparison guides and campaign pages. Cookie policy