Package 012 compute chips, 2 memory stacks
Liquid cooled, OAM moduleSession memory, writableCompute, ready at power-on

Model-specific inference silicon

Inference,cast insilicon.

A purpose-built inference rack for B2B data centers. Your model becomes part of the hardware, so every token comes back faster, costs less energy and leaves nothing in the field to tamper with.

Discuss your workload

Drag the chip to rotate it

Bit-exact full-chip simulation on real model weightsSimulation result

Five-chip stack with session memory, validatedSimulation result

16,300 tokens/s for one user on a 1T-parameter modelEngineering estimate

Scroll

01The bottleneck

A GPU re-reads the model for every word.

It does this for every token and every user. Batching hides the cost on a utilization chart, but each person waiting on an answer still feels it. Race one answer and see.

Typical GPU, per user

0.0sInfer18, per user

0ms

Engineering estimateGPU lane at about 100 tokens/s, a common per-user speed on large models. Infer18 lane at about 16,300 tokens/s on a trillion-parameter-class model.

0.0 GBModel data a GPU moves for this one answer

0 BMoved by Infer18

Simulation resultA compact 270M-parameter model re-reads 268 MB per token per user. Larger models move far more.

Speed for one user, trillion-parameter-class modelEngineering estimate

02From a full cluster to one chip

Choose a level

01 / 06Engineering estimate

Cluster

Racks joined on your Ethernet spine. The largest models span racks; smaller ones run many copies in one. Keep scrolling to take it apart.

Llama 3 70B8 copies per rack

Llama 3.1 405B4 nodes per copy

Kimi K29 nodes per copy

GrowthAdd racks as volume grows

03Efficiency

Fast for one person. Efficient for millions.

Pick a model. Each square is one node of Infer18 serving one full copy of it.

Speed for one user16,300Tokens/s with no batching, at least 54× a GPU’s best per-user speedEngineering estimate

Ceiling per copy4.1MTokens/s across every user of one copyEngineering estimate

Long chats held18KOpen 8,000-token conversations per copyEngineering estimate

500-token answer31 msGeneration time inside the rackEngineering estimate

Long chats stay fastEngineering estimate

Speed on an 8B-class model as a conversation grows.

Conventional attention6,100 tok/s

Infer18 long-context engine9,390 tok/s

1.5×

Scale a fleetEngineering estimate

Chat messages a month on one compact model:720 billion

5,840 H200-class GPUs6.4 MW

271 Infer18 server chips42 kW

The amber band is the GPU fleet's power draw and the blue line is Infer18's: about 150 times less for the same traffic, from a worked example of 720 billion messages a month.

04Security

Nothing to tamper with. The model is the chip.

On a GPU server the model is a file. Anyone who can reach that file, whether an attacker, a bad update or a wrong upload, can swap it, alter it or quietly poison it, and the server will serve the changed model without noticing.

On Infer18 the model is fixed into the chip when it is made. There is no file to replace and nothing to rewrite, in the data center or anywhere on the way there.

On a GPU serverA changed model goes live, unnoticed.

On Infer18Nothing to change. The original model, every time.

Proposal: security properties follow from the design and await silicon validation

05Fits your data center

Beside your GPUs, or on its own.

Send a request

Users and agentsRequests in

Your routerOpenAI-compatible API

GPU clusterTraining, new models

Infer18 racksStable, high volume

Your flagship model is built into Infer18, so the router sends it to the racks and the answer streams back at full speed.

Four ways to split the work

Your GPUsNew models, fine-tunes, the long tail and training

Infer18Every request for your highest-volume stable model

What you gainGPUs freed for training, and far fewer watts per token on your busiest traffic

Will it fit your hall?Tick what you already have.Liquid coolingDirect-to-chip, as in current GPU racks, at about 50 kW per rack.54 V DC busbar and power shelvesThe same power layout as modern GPU racks.400 GbE Ethernet spine with RDMAJoins your existing spine. No NVLink needed.An LLM request routerRegisters as one more pool behind it.Installs like a GPU rack.

06Proof

Every claim, labelled.

Simulation results, engineering estimates and proposals are kept apart across this site. Below, each test is harder than the last, and every generated token must match the reference model exactly.

Simulation resultEngineering estimateProposal

Bit-exact tests, each harder than the lastSimulation result

One chipA complete compact model on a single chip, given a 600-word prompt.

24 / 24tokens identical to the reference model

Five-chip stackA 1B-parameter model split across five chips that hand work to each other.

8 / 8tokens identical across the chip-to-chip hand-offs

Stack + session memoryThe same five chips, with conversation memory kept off-chip.

8 / 8tokens identical, only 0.3% slower than on-chip memory

0mismatches across 917 attention tests on real model dataSimulation result

14 / 14model architectures compiled to bit-exact chipsSimulation result

0.4%from the original model’s quality in exact modeSimulation result

+0.3%time cost of moving session memory off-chipSimulation result

Compiled to bit-exact chipsLlama, Gemma, Mistral, Mixtral, Qwen 2, Qwen 3, Qwen-MoE, DeepSeek V3, OPT, GPT-2

07Is it right for you

Built for the models you run all year.

The model will stay in service for a year or moreIt serves high, steady token volumeIt is open-weight, or licensed for hardware

Strong fitStart with a pilot rack beside your GPUs and move your busiest model onto it.

Build your cluster Request a pilot rack