← Back to the Lab

A desktop tower saved from Windows 10's end of support, now running local inference.

The Local AI Machine

When Microsoft announced the end of support for Windows 10, my desktop tower suddenly loomed toward obsolescence. Rather than send it to the e-waste pile, I turned it into a Linux machine — but a plain daily driver would’ve been redundant, since I already run an M1 MacBook Pro for that.

Why local

Meanwhile, when DeepSeek R1 released and briefly rattled the market, DeepSeek open-sourced the model. That got me wondering whether I could run it myself and skip handing my usage over to an AI company’s data collection.

The build (and the bottleneck)

I picked up an EVGA GeForce RTX 3080 FTW3 — only to find out afterward that I was technically bottlenecking my own speeds due to a PCIe protocol mismatch. Worth admitting, but not worth pretending around.

But instead of using it for gaming like a normal person, I dove down the rabbit hole of self-hosting AI. Ollama as the backend, Open WebUI as the interface, and sorting through the sprawl of “local models” available — including realizing how many of them turn out to be Alibaba’s Qwen underneath with a different system prompt.

The model roster

  • HuiHui DeepSeek R1 Abliterated (Qwen) — daily driver, deliberately uncensored
  • Qwen 3 – general use
  • Microsoft Phi 4 — technical and research work for engineering questions
  • Google Gemma 4 — Google’s take
  • Qwen 3 Coder — coding
  • Nomic Embed Text — text embedding for retrieval-augmented generation (RAG)

What’s next

Fine-tuning/personalization, along with wiring in real web search — Open WebUI paired with SearXNG has turned out to be more finicky to configure than expected.

Also: image generation using ComfyUI and FLUX.1-Schnell. Proof of concept only: I have no desire to replace human art.

Questions You Might Be Asking

Why not just pay for a hosted AI subscription?+

A few reasons: 1. I have an ever-growing aversion to subscriptions. You might notice that pattern in other projects I'm taking on. 2. I prefer to keep my data to myself. I'm not opposed to sharing data, but I like having the option to share what I want. 3. Environmental concerns: reducing my personal consumption makes me feel slightly less guilty of taking up clean water (my tower is air-cooled) or electricity (I don't think I need beyond 1kWh even to help me solve a distillation problem). 3. If nothing else, I like having local AI as a proof of concept.

Isn't bottlenecking a 3080 kind of embarrassing to admit to?+

Yes. In my defense though, this was before I had any idea about how to match hardware. I just bought the best GPU I could afford and paired it with the tower I had already built.

How did you decide on models?+

When I first started the project, I was browsing Reddit for whatever local models would be nice to have. I started with DeepSeek, then an abliterated version of DeepSeek (to get around political censors), then a mixture of experts (MoE) model (I chose Mistral's Mixtral), then GPT-oss, and before I knew it, I was flooded with multiple models. Then, during a conversation with Gemini on fine-tuning, I learned that many of the models were based on Alibaba's Qwen with system prompts. I peeled through the models, removed basically everything that was based on Qwen (with the exception of local DeepSeek R1), and limited myself to all distinct models. From here, I'm working on fine-tuning my own models by giving system prompts and knowledge bases (chat history from ChatGPT, Gemini, and Claude).