A full-size GPU in a little box,
answering from a desk.
Forge is a local LLM chat running entirely on an NVIDIA RTX 5070 in a Razer Core X V2 eGPU enclosure, connected to a laptop over USB4. No cloud inference. When the desk machine is on, invited users can chat with it here — and watch the card work, live.
What it's doing
You type a prompt. It crosses HTTPS to a small proxy, rides a private tunnel down to the desk machine, and lands in Ollama, which runs the model on the 5070. Tokens stream back the same way, one at a time — about every 10 ms at full tilt. Next to the chat, a live rail shows what the hardware is actually doing: watts, utilisation, VRAM, temperature and clocks, polled from nvidia-smi twice a second. The point of the page is the pairing: words appearing because a graphics card on someone's desk is pulling 180 watts.
The stack
| GPU | NVIDIA GeForce RTX 5070 (12 GB, Blackwell, compute 12.0), driver 621.14, in a Razer Core X V2 over USB4 / PCIe Gen 4 x4 |
| Model | Qwen3 8B (Q4_K_M GGUF, ~5.2 GB) via Ollama. Thinking mode off for chat, so replies stream immediately. |
| App | Python standard-library server. No framework, no pip installs: it proxies Ollama's streaming API and republishes GPU telemetry as server-sent events. |
| Path to you | Caddy on the DataRail VPS terminates HTTPS and reverse-proxies to the desk machine over a private SSH tunnel that the desk machine dials out. Nothing on the home network is exposed inbound. |
| Honest limit | Forge is online only while the desk PC and eGPU are on and awake. If they're off, you get an “offline” page, not a silent spinner — check the pill at the top of this page. |
Access
Invite-only. There is no self-signup. Accounts are created by the operator, passwords are stored as scrypt hashes, sessions are hashed tokens in an HttpOnly cookie (7 days), and five failed sign-ins lock an address for ten minutes. Chat and telemetry refuse every request without a session — only this rundown and a minimal online/offline check are public. To ask for an account, write to contact@datarail.org.
Why build it
As a proof that “AI hardware” doesn't have to mean a datacentre rental. A current-generation consumer card in an external box, on a laptop-class machine, sustains ~100 tokens/sec on an 8B model — faster than most people read — while drawing less power than a kettle. Forge is the demo you can point people at: a link, a login, and a power meter that proves where the work happened.