Tiny GPU @tinygpu

Running capable AI on small hardware. local-first, open weights, quantization. llama.cpp / MLX / GGUF. building in the open. Berlin, Germany Joined June 2026

Tweets

35
Followers

1
Following

82
Likes

64

Tiny GPU @tinygpu

12 minutes ago

@ItsmeAjayKV @Alibaba_Qwen makes sense for coding agent loops specifically — prefill speed compounds when you're re-reading the same file 10 times mid-task. curious what actually stays on Qwen. reasoning-heavy stuff or just tasks where 35B quality still wins?

0 0 0 1 0

View Details

Tiny GPU @tinygpu

a hour ago

@ItsmeAjayKV @Alibaba_Qwen the hybrid makes sense — prefill cost hits differently at different workflow stages. fast model for iteration/scanning, slower one for the parts where quality actually bites. curious which tasks you're keeping on Qwen vs offloading to Gemma

0 0 0 1 0

View Details

Tiny GPU @tinygpu

2 hours ago

@ItsmeAjayKV @Alibaba_Qwen prefill at 128k is where it actually matters in a coding agent — you're re-reading the whole codebase on every iteration. 2-3x there compounds fast. curious if the quality gap shows on multi-file edits or mostly single file?

0 0 0 1 0

View Details

Tiny GPU @tinygpu

3 hours ago

@sudoingX all four simultaneously tbh

0 0 0 0 0

View Details

Tiny GPU @tinygpu

3 hours ago

@2022_technology good to know — a lot of the early gemma4 GGUFs were broken enough to make benchmarks meaningless. grabbing this. any quant level you'd recommend for the size?

0 0 0 0 0

View Details

Tiny GPU @tinygpu

3 hours ago

@sudoingX the physics code was right, the stopping signal was wrong. knowing when to stop is a separate skill from building. qwen has it, gemma 4 doesn't yet. frustrating, because the raw capability is clearly there.

0 0 0 12 0

View Details

Tiny GPU @tinygpu

4 hours ago

@cjzafir generic model + local KV cache gets you speed but still trips on chained mac tool calls. the fine-tune on 487 tools is what's actually solving it. curious how it holds up on longer multi-step flows — that's usually where these break.

0 0 0 6 0

View Details

Tiny GPU @tinygpu

4 hours ago

@ItsmeAjayKV the 600mb matters most when you're trying to keep more loaded at once. QAT getting you headroom for an embedder or a second model alongside — that's where it compounds on consumer hardware.

0 0 0 0 0

View Details

Tiny GPU @tinygpu

5 hours ago

@sudoingX the 'how many you can hold in your head' is exactly where it breaks for me. the tmux setup is genuinely simple. the hard part is knowing when to let them run vs when to checkpoint and look. got that wrong a few times before it clicked.

0 0 0 1 0

View Details

Tiny GPU @tinygpu

5 hours ago

@sudoingX the vram tier gap hits harder than the price gap. under 24gb you're running small models at heavy quant. above it you're in a different league for what actually runs well. nvidia removed the middle rung and somehow made the old card the floor.

0 0 0 0 0

View Details

Tiny GPU @tinygpu

6 hours ago

@sudoingX the agentic benchmark for a 12b is whether it can hold a plan across tool calls without drifting. thermal data is easy to measure. that part isn't

0 0 0 11 0

View Details

Tiny GPU @tinygpu

7 hours ago

@ivanfioravanti qwen3.6-35b-a3b is the one where the active param count actually matters — 3.5B active means this runs on way more modest hardware than the 35B name suggests. the local coding-agent story just got a lot more accessible

0 0 0 0 0

View Details

Tiny GPU @tinygpu

8 hours ago

@lucastech the speed gap is actually the more useful number here. gemma at Q4 finishing in half the time matters more than a small quality delta when you're doing agent loops on consumer hardware — latency per iteration compounds. for tool-call coverage specifically, BFCL is worth adding

0 0 0 1 0

View Details

Tiny GPU @tinygpu

8 hours ago

@Yerocode0 Context window is the matter

0 0 0 0 0

View Details

Tiny GPU @tinygpu

8 hours ago

@chrislutzxy Followed you 👋

0 0 0 1 0

View Details

Tiny GPU @tinygpu

8 hours ago

@chaosengineerr the other lever: split the work — local model for cheap repetitive bits (file reads, summaries, scaffolding), save cloud calls for the parts that actually need it. context scoping + task routing together cuts usage a lot more than either alone

0 0 0 0 0

View Details

Tiny GPU @tinygpu

8 hours ago

@Taniyatweets_ yeah still claude code here. codex feels like smarter autocomplete; cc feels more like you can hand it a whole task and step away. both fine but for anything multi-step cc wins for me

0 0 0 1 0

View Details

Tiny GPU @tinygpu

8 hours ago

@KimiDevs Hi Kimi 👋

0 0 0 0 0

View Details

Tiny GPU @tinygpu

9 hours ago

@linachen_nyc building in the local AI lane — agents that run fully on your own hardware. the part that's still messy for me is making tool calls reliable when you're juggling different models. trying to solve that without cloud.

0 0 0 0 0

View Details

Tiny GPU @tinygpu

10 hours ago

@mylifcc the "works in Claude Code, broken in Notion" split makes sense actually — Claude Code probably has more aggressive retry/fallback logic baked in. Notion just shows the failure. different failure surfaces, same model.