How Much Does It Actually Cost to Run a Local LLM? (Euros per Million Tokens, Measured)

The analysis dives into the real-world costs of running various local language models on a single RTX 3090 GPU, revealing surprising insights about efficiency. Contrary to expectations, the smallest model wasn't the most economical, and the largest one didn't necessarily bear the highest costs. This study sheds light on the financial implications for businesses and researchers considering local versus cloud-based AI solutions, emphasizing the importance of understanding actual resource usage and costs to make informed decisions.