Local AI
Running open models on my own hardware and exploring how they connect to tools.
I designed and assembled a system to explore what I can do with AI models running locally. I use it for coding assistants and voice-to-text utilities, and I experiment with fine-tuning and connections between models and tools.
The system
The system uses an AMD Ryzen 9 9950X3D, 64 GB of DDR5 memory and a Radeon RX 9070 XT. The largest constraint I have encountered is VRAM, which limits the models I can run efficiently. For my use, I consider a model efficient when it generates more than 15 tokens per second.
Running and fine-tuning models
For coding, I use Qwen 3.8 27B and Gemma 4 12B, while my voice-to-text utilities use various models. The tools I have experimented with include Ollama, LM Studio, llama.cpp, ComfyUI and Unsloth Studio.
I also fine-tuned Microsoft Phi-4 mini. This experience showed me that training a model is a difficult task, and that it is easy to over-train a model if I am not careful.
Connecting local agents
I put together a Model Context Protocol (MCP) connection between Antigravity and local agents running through Ollama and LM Studio. Through this connection, Gemini models can use the local agents as part of a workflow.
Connecting local agents in this way gives me an opportunity to explore how models work with tools and with each other.