I use big LLMs to draft blog posts and LinkedIn posts. They are great at research and structure, and terrible at sounding like me. Every draft comes out with the same polished, slightly corporate voice, and even after I edit it by hand, something of that voice stays in the text. There is actually research confirming this: a 2026 study on post-editing found that even when humans edit LLM drafts, the result stays measurably closer to LLM style than to the author’s own writing.
So I decided to test an idea: keep the big model for the heavy lifting (research, structure, first draft), and fine-tune a small local model to do one job only – rewrite the draft in my voice. This post is about what it took to get there on a single RTX 3060, and everything that broke along the way.
Why this should work at all
Before spending a weekend on it, I checked whether fine-tuning on a few hundred examples has any chance. Turns out the evidence is pretty strong:
- LIMA showed that ~1,000 curated examples are enough to align a model, because small-data fine-tuning teaches style and format, and knowledge stays in the base model.
- Panza built personalized email assistants from under 100 emails. In their blind study, over two thirds of generated emails were rated as credibly human.
- TinyStyler beat GPT-4 on authorship style transfer with an 800M parameter model.
- A tone-of-voice study got a 1B model to match a target style with only 100 samples, beating system-prompting baselines.
The style curve saturates somewhere between 25 and 100 good examples. The hard part is a different one: where do the examples come from?
Step 1: Finding 30,000 words of my actual writing
The uncomfortable discovery: I barely have any. My blog has posts going back to 2011, but everything from 2023 onward was written with AI assistance to some degree, so it can’t be ground truth for my voice. After stripping code blocks, the clean 2015-2018 posts came out to about 2,900 words of prose. That is nothing by fine-tuning standards.
So I went digging for places where I actually type:
- Gmail Takeout. An 11GB mbox parsed with Python’s built-in
mailboxmodule. Keep only messages where I am the sender, strip quoted text and signatures, keep English messages with 60+ words of real prose. After filtering: ~15,000 words. - Beeper. Beeper’s desktop app keeps all bridged chats in a local SQLite database (
~/.config/BeeperTexts/index.db), with a convenientisSentByMeflag. My work Slack turned out to be the best source here. One trick that tripled the yield: merging message bursts – consecutive messages I sent in the same chat within 5 minutes – into a single sample, because I tend to type several short lines instead of one long one. ~15,500 words. - Agent session logs. Claude Code and Codex keep full session transcripts as JSONL on disk. My own typed prompts in there are authentic first-person writing too. After filtering out pasted documents and code: ~11,000 words.
Everything is processed locally. Nothing leaves the machine, which is the whole point of a local fine-tune.
One warning if you try this with your own mailbox: automated garbage is everywhere. My first pass happily included Google Calendar invitation boilerplate as “my writing”. Filter aggressively.
Step 2: Manufacturing training pairs by de-styling
A rewriter model needs pairs: a bland draft as input, my text as output. I have the outputs. Nobody wrote boring versions of my emails to serve as inputs.
The trick is to generate the inputs backwards, and it has a name in the literature: inverse paraphrasing (STRAP), or “reverse instructions” in the Panza paper. For every piece of my real writing, a fixed model (always the same one, for consistency) rewrites it as a neutral, personality-free draft with the same facts. Then the pair goes into training reversed: neutral draft in, my original out.
A real example from my corpus. My 2015 blog post said:
Long story short: … I tested it personally and it worked like a charm.
The de-styled draft the model trains against:
To summarize the process: … This was tested directly and worked without issue.
Multiply by ~150 and the model learns the direction of the transformation: generic polish in, my cadence out.
The pairs are formatted as chat SFT – system prompt says “you are Ann’s voice rewriter”, user message carries the draft plus a register tag (blog / email / chat), assistant message is my original. Held-out split is by document, so no chunk of a test document ever leaks into training.
Step 3: Training on the 3060
The stack is Unsloth + TRL’s SFTTrainer, QLoRA at rank 16 on all attention and MLP projections. The base model question had a hardware answer:
- Gemma-3-12B was my first pick (the community favorite for prose). In 4-bit it needs ~7GB of VRAM for weights alone, and with my desktop session already holding 2.5GB of the card’s 12GB, the loader refused – bitsandbytes will not train with layers offloaded to CPU. It fits only on a bare GPU.
- Qwen3-8B in 4-bit sits at ~5.5GB and trains comfortably next to a running desktop.
With ~150 pairs, 5 epochs and early stopping on held-out loss, a full training run takes 16-20 minutes on the 3060. That number changed how I worked: at 20 minutes per experiment you stop being precious about runs and start iterating like it’s a unit test suite.
For the record, the config that finally behaved:
model = FastModel.get_peft_model(
model,
r=16,
lora_alpha=32,
lora_dropout=0.05,
target_modules=["q_proj", "k_proj", "v_proj", "o_proj",
"gate_proj", "up_proj", "down_proj"],
use_gradient_checkpointing="unsloth",
)
# in SFTConfig: learning_rate=1e-4 (more on that below),
# per_device_eval_batch_size=1, prediction_loss_only=True
That last pair matters: the eval defaults will happily OOM a 12GB card converting full logits to fp32.
Everything that broke
This is the part I wish someone had written before me.
My training data was poisoned by my own tooling. I generated the de-styled drafts with headless claude -p calls. What I did not notice: the calls inherited the project’s plan mode, and 48 of 173 drafts began with meta-commentary like “This is a simple text rewriting task, not a coding task…”. The model trained on that junk, and run 1 would sometimes respond to a rewrite request with reasoning about the task instead of the rewrite. Lesson: read your generated training data. All of it, or at least a real random sample.
Run 2 collapsed into a copy machine. After cleaning the data I added a “keep the same length, do not add content” instruction and kept a conservative learning rate of 1e-5. The result faithfully returned the input, word for word. Copying was the cheapest way to satisfy the loss. Raising the LR to 1e-4 (the normal LoRA range – 1e-5 is a full-fine-tune number) and softening the instruction fixed it.
The OOM killer, twice. The kernel killed training at the exact moment weight loading finished. There was plenty of free VRAM, so I stared at CUDA for a while before checking free -h: a browser was eating 10GB of host RAM and zram swap was already 60% full. If your training dies with no Python traceback right after the weights load, look at host memory first.
Boilerplate contamination. Calendar invites, HTML-heavy forwarded newsletters, and pasted AI output inside my own emails all made it into early corpus versions. Each one needed its own filter.
Honest results so far
After four runs, the picture is clear and register-dependent.
Email and chat: it works. A de-styled draft comes in as “Hello, apologies for the delayed response – this week was quite busy… Before running it, execute npm install” and comes out as “Hello, sorry for the delay, this week was very busy… Before running it, run npm install to install the dependencies.” Lowercase tech terms, comma splices, no corporate connectors. Facts intact. It reads like me on a normal day.
Blog: does not work yet. The model mostly returns blog drafts untouched. The reason is boring and structural: only 20 of ~150 training pairs are blog register, because that’s all the clean blog prose I have. The fix in progress is hand-curating my AI-assisted 2023+ posts back into my real voice, which produces the most valuable training pairs possible – real (AI draft, my edit) examples, exactly the distribution the model will see in production.
Two caveats I want on record. First, this will make text credible to people who know me; forensic stylometry still wins against every current personalization method, which is fine, since I’m not trying to fool anyone – the byline is mine either way. Second, I have not yet run the honest benchmark against my existing setup (a detailed style guide plus few-shot examples in the big model’s prompt). Practitioner reports are split on whether fine-tuning beats strong prompting, so the deciding test is a blind comparison where I rank outputs without knowing which is which. If the fine-tune only ties, the writeup will say so.
Wrapping Up
The pipeline works end to end: corpus extraction from mail, chat and agent logs, pair synthesis by de-styling, QLoRA training in 20-minute iterations on a 12GB consumer card. The registers with enough data already sound like me. The register I actually started this for – blog posts – is still waiting on better data. As usual, most of the real work turned out to be data work.
Next steps: finish curating the blog pairs, retrain, run the blind eval, and if the fine-tune earns its keep, wire it into my drafting workflow as a local llama-server behind the big model.
Everything here ran locally on Arch Linux, an RTX 3060 12GB, 32GB RAM. Model weights and checkpoints live on a separate drive because the home partition filled up mid-project.. but that’s another story.