This little Mac made me rethink local AI

What 16GB, 32GB, and 512GB reveal about running a model on hardware you control.

Apple’s new Mac mini grows yellow clay arms, does push-ups, becomes a race car, and launches through handmade clouds. The whole film lasts 43 seconds. I thought it was super cool. Partly because the little machine looks ridiculous. Mostly because the making-of lets you see the human work: clay bending, rigs holding the models, and each movement built one frame at a time. Stop-motion animator Kevin Parry’s “Umm who made this” post crossed two million views. A making-of clip shared by Andreas Storm crossed 176,000. He had a point. TBWA\Media Arts Lab developed the campaign; Blinkink and Stoopid Buddy Stoodios produced it; Nicos Livesey directed it. But the film is not why I got carried away. Apple’s announcement gives the little machine a much more serious job. The company now describes the Mac mini as an “always-on, deskside” computer for agentic AI. Last week, I wrote that the model should be replaceable while the worker’s memory, permissions, routines, and audit trail stay. This launch made me think about the other side of that sentence: where should the model itself run? Eight days before Apple’s launch, OrcaRouter published an Apple Silicon build of Qwen3.8-27B with a very clean promise: “No CUDA. No cloud. Just your Mac and the weights.” My first reaction was just as clean: This is it. AI is becoming fully decentralized. I was getting ahead of myself. The model can run locally. That part is real. But “local” does not mean “any model on any Mac.” The useful 4-bit files are about 15GB. Apple’s new $899 Mac mini starts with 16GB of unified memory, which sounds close enough. It is not. OrcaRouter lists 24GB as the minimum and calls 32GB the practical choice for LM Studio. The weights are only part of the memory bill. macOS, the application, the model runtime, the vision components, and the working context need room too. The 2-bit version is smaller, at about 8.7GB, but the uploader calls it severely degraded and says not to use it for real work. So “just your Mac” needs an asterisk. If I were buying a machine for this specific model, I would start at 32GB for the 4-bit build. I would choose 64GB if I wanted the higher-quality 8-bit version and more room for context and other applications. I would not buy a 512GB Mac Studio to run a 27B model. That machine solves a completely different problem. The M5 Ultra Mac Studio can be configured with 512GB of unified memory and 1.2TB/s of memory bandwidth. Apple says that configuration is coming in late October. It is enough capacity for much larger models, several models at once, or a heavy creative and AI workload on the same machine. One correction matters here: 512GB is not new. Apple already offered it with the previous M3 Ultra. The new part is the M5 Ultra system around it. Apple is also selling local AI much more directly as a job for the Mac. People immediately started comparing the 512GB Mac Studio with NVIDIA’s DGX Spark. On paper, the top Mac has four times Spark’s memory and more than four times its published memory bandwidth. That does not make it four times faster. DGX Spark has 128GB of unified memory, CUDA, ConnectX networking, and purpose-built AI compute. The Mac has a much larger maximum memory pool and the MLX stack. Memory tells me what can fit. Compute, precision, context, and software decide how well it runs. Without the same model, precision, and workload running on both machines, there is no honest winner yet. I also gave the word “uncensored” too much credit. Qwen3.8-27B is the official open-weight base model from Alibaba’s Qwen team. The “Uncensored MLX” version is OrcaRouter’s community derivative. It uses a technique called abliteration to reduce refusal behavior by removing a learned refusal direction. That does not make the model more truthful. The model card warns that it may comply with harmful requests and state false things confidently. Its “zero refusals” and quality claims are the uploader’s own tests, not an independent evaluation. Less refusal is not the same as more intelligence. My read: the real shift is being able to control where the inference happens. With enough memory, the weights, prompts, documents, and generation can stay on a machine you control. A hosted inference provider does not need to sit inside every request. Local inference can keep running through a provider outage. But that only stays private if the application, tools, telemetry, and logs stay local too. And the responsibility moves with it. You own the updates, backups, access control, evaluation, and safety layer. Calling that fully decentralized overstates it. Apple still controls the hardware and operating system. Qwen creates the base model. Hugging Face distributes the files. MLX and the applications around it still depend on maintainers. The useful gain is still real: you get to decide which parts of the AI loop stay with you. Before I call a setup “local,” I want three answers: Which model and precision am I actually going to run? Does the machine have room for the weights, runtime, and real working context? Which applications, tools, and logs still send data somewhere else? I am still excited about this, but for a more honest reason now. Local AI is becoming a normal hardware decision. You can start with the work, choose the model, calculate what it needs, and decide whether that work should ever leave your desk. That feels much more real to me than another “four times faster” launch claim. What is one piece of work you would want to keep completely on a machine you control?

Harshith Vaddiparthy works with founders, operators, and teams on practical AI products, workflows, advisory, training, and mentorship. This no-JavaScript version preserves the page's core information and navigation.