Running 177B MoE Models on a 12 GB GPU with llama-moe-cache and NVMe Streaming [Part 3]

I wanted to try Qwen 3.8 Flash Next, a 177B MoE model, on the same RTX 3060 used in Part 1. Its quantized weights occupy 76.3 GiB, and my machine has 12 GB of VRAM and 32 GB of system RAM.

The first launch froze my desktop. Getting it working required changing how the server loaded the weights.

I used the UD-IQ3_XXS quantization and GenerelSchwerz’s llama-moe-cache branch to test GPU expert caching with weights backed by NVMe storage.

[Read More]

Benchmarking Terminal Coding Agents: 920 Public Trials and 460 Private Trials [Part 2]

This is Part 2 of a three-part series on running, benchmarking, and scaling local coding agents. Part 1 covers the local Qwen setup; Part 3 explores running a larger MoE model on the same desktop hardware.

In Part 1, I got Qwen 3.6 35B A3B NVFP4 running locally on an RTX 3060 with a 64k context window and wired it into Crush, Pi, and Oh My Pi. I wanted to know how that setup compared with the terminal coding agents I already had on subscriptions. Could it finish the same tasks, and would I want to merge the patches?

[Read More]

Running Qwen 3.6 35B NVFP4 Locally with FreeToken: Fitting a Coding MoE into 12GB VRAM [Part 1]

This is Part 1 of a three-part series on running, benchmarking, and scaling local coding agents. In Part 2, we benchmark this setup across 640 public trials and 280 private trials on a production Go monorepo, including Claude Code, Codex, Antigravity, local Qwen, and two hosted models. In Part 3, we build and test the specialized llama-moe-cache fork to run 177B Qwen3.8-Flash-Next on desktop hardware.

I wanted a capable local reasoning model for coding agents on my workstation, but my GPU is a standard desktop NVIDIA GeForce RTX 3060 with 12GB VRAM. The weights would not all fit in VRAM, so I needed a setup that could offload them.

[Read More]

Fine-Tuning a Local LLM to Write in My Voice: QLoRA on an RTX 3060 with Unsloth

I use big LLMs to draft blog posts and LinkedIn posts. They help with research and structure, but I still spend time rewriting them to sound like me. Every draft comes out with the same polished, slightly corporate voice, and even after I edit it by hand, something of that voice stays in the text. There is actually research confirming this: a 2026 study on post-editing found that even when humans edit LLM drafts, the result stays measurably closer to LLM style than to the author’s own writing.

[Read More]

Arch Linux: Hyprland 0.46.2 Crashes After Upgrade — Here's a Temporary Workaround and a hyprutils-git Conflict Fix

Yesterday I encountered a nasty surprise after upgrading Hyprland to version 0.46.2—it crashed on startup, leaving me unable to log into my Wayland session. If you’re experiencing this problem, here’s a temporary workaround so you can keep working until a fix arrives.


The Issue

Hyprland 0.46.2 appears to crash immediately upon launch, preventing you from getting into your usual tiling window manager environment.


Update Dec 29 2024: hyprutils-git Caused Compatibility Issues

After some further digging, I discovered my Hyprland crash was caused by the AUR hyprutils-git package. I had previously switched to hyprutils-git from the AUR because I was also using hyprpolkitagent-git. Because of this, the entire Hyprland ecosystem ended up depending on the hyprutils-git package, which broke compatibility with the stable release of Hyprland in the last few updates.

[Read More]

Hyprland Quick-Start Guide Part 2: Enhancing Your Setup with Advanced Features

Update September 2026: I updated these examples after moving my CachyOS setup to Hyprland 0.56.2 and Lua. I switched to rofi, added swaync controls to Waybar, and started sorting out what happens to my workspaces when I unplug the dock. For the older compositor syntax, see the archived hyprlang docs.

In Part 1 we set up the desktop. Let’s put those configs under Stow first so we can keep them together while we change the theme and add the other helpers. The startup examples here go inside the same Lua hook.

[Read More]

Hyprland Quick-Start Guide: Setting Up a Modern Tiling Window Manager on Linux

Update September 2026: I revisited this setup on CachyOS with Hyprland 0.56.2. I moved the compositor config to Lua, switched the launcher to rofi, and got my wallpaper back with the new hyprpaper config. Some changes produced config errors. Others left a process running that did nothing, which took longer to figure out.

We’ll set up Hyprland with a terminal, file manager, launcher, bar, and lock screen. These examples target the setup I revisited. If you’re following the original guide on Hyprland 0.54 or older, use the archived hyprlang docs.

[Read More]

Local Dev Craft: A Dive into Kubernetes with kind, KEDA, and LocalStack AWS (Part 1)

In this first post of my new Local Dev Craft series, we will be setting up a local Kubernetes cluster using kind, KEDA, and LocalStack AWS in order to build a development/debugging environment that is similar to production. But first, let’s get familiar with the tools we are going to use here:

  • Kubernetes: An open-source platform designed to automate deploying, scaling, and operating application containers.
  • kind (Kubernetes in Docker): A tool for running local Kubernetes clusters using Docker container “nodes”.
  • KEDA (Kubernetes Event-Driven Autoscaling): A set of components that extends Kubernetes to provide event-driven autoscaling for every container.
  • LocalStack AWS: A fully functional local AWS cloud stack for testing and mocking AWS services locally.

Now that we know what we’re dealing with, let’s move on and create our local development environment that mimics our cloud setup. This setup will be a set of stateless jobs that will be spawned by KEDA events, exit after execution, and scale to zero.

[Read More]
Categories: local-dev  Tags: local-dev kubernetes kind keda aws localstack