Back to feed
Dev.to
Dev.to
6/23/2026
How to Rank Local LLMs by Cost per Correct Answer (Measured GPU Energy, 8 Ollama Models)

How to Rank Local LLMs by Cost per Correct Answer (Measured GPU Energy, 8 Ollama Models)

Short summary

Benchmarking 8 local Ollama models by cost-per-correct-answer using real GPU energy metering, gemma4:26b wins at €0.013/1k (96.9% accuracy) while qwen3:8b-fp16 costs 18× more for worse results. Reasoning tokens and full precision didn't improve accuracy on deterministic tasks. HomeLab Monitor provides open-source infrastructure to measure your own models' cost-per-correctness.

  • •Gemma4:26b achieves 96.9% accuracy for €0.013 per 1,000 correct answers — 18× cheaper than qwen3:8b-fp16
  • •Reasoning tokens and full-precision quantization added cost without improving accuracy on structured tasks
  • •HomeLab Monitor (open-source) measures real GPU power consumption to calculate cost-per-correctness for any model

Generated with AI, which can make mistakes.

#ai-tools#market-trend#research-breakthrough
Read full article at Dev.to

Is this a good recommendation for you?

Explore more

Model Showdown Round 3: Ditching Ollama in Favor of llama.cpp
Dev.to
AI model cost chart shows
Dev.to
I shipped local LLM features two months ago. Production never ran them once.
Dev.to
How much VRAM do you actually need to run Llama 3 or Gemma locally?
Dev.to