Your GPU, Your Model: Running Open-Weight AI

Tired of cloud API latency, rate limits, and per-token bills? Open-weight models have gotten good enough, and runtimes fast enough, that your workstation can do real work. This session covers how to run, quantize, and serve open-weight models on your own hardware, and where the limits are. We’ll start with what a model actually is (weights, parameters, tokens) and why running it locally makes sense (or doesn’t). Then we’ll get practical: Quantization: choosing a method and bit width for your memory budget, and what you give up Inference engines: picking and configuring the right runtime for your hardware and use case Hardware sizing: rules of thumb for memory bandwidth, VRAM, and context window management on local silicon You’ll leave knowing what your machine can realistically run, how fast, and when the cloud is still the better call. Bring your questions.