<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom">
	<channel>
		<title>Local-Llm on Dr. Hitesh Sarda</title>
		<link>https://hitesh.in/tags/local-llm/</link>
		<description>Recent content in Local-Llm on Dr. Hitesh Sarda</description>
		<generator>Hugo</generator>
		<language>en-US</language>
		
		
		
		
			<lastBuildDate>Tue, 30 Jun 2026 00:00:00 +0000</lastBuildDate>
		
			<atom:link href="https://hitesh.in/tags/local-llm/index.xml" rel="self" type="application/rss+xml" />
			<item>
				<title>Local models stopped being toys</title>
				<link>https://hitesh.in/2026/local-models-stopped-being-toys/</link>
				<pubDate>Tue, 30 Jun 2026 00:00:00 +0000</pubDate>
				<guid>https://hitesh.in/2026/local-models-stopped-being-toys/</guid>
				<description>&lt;p&gt;For two years my default advice to anyone asking about AI tools for development has been: pick a cloud API, learn to use it well, and don&amp;rsquo;t overthink the hosting question. The local-model path (downloading weights, fiddling with quantization levels, doing the RAM math) looked like a distraction from the actual work.&lt;/p&gt;&#xA;&lt;p&gt;That advice is starting to age.&lt;/p&gt;&#xA;&lt;h2 id=&#34;what-shifted&#34;&gt;What shifted&lt;/h2&gt;&#xA;&lt;p&gt;A dense 27B model is &lt;a href=&#34;https://quesma.com/blog/qwen-36-is-awesome/&#34;&gt;now hitting 30 tokens per second&lt;/a&gt; on M5 Max hardware and doing genuine coding work, the kind that survives code review. Qwen 3.6 27B is generating real enthusiasm among developers, and the reason is straightforward: it works on real tasks.&lt;/p&gt;</description>
			</item>
			<item>
				<title>The real cost of running AI locally for coding</title>
				<link>https://hitesh.in/2026/local-ai-coding-cost/</link>
				<pubDate>Tue, 16 Jun 2026 00:00:00 +0000</pubDate>
				<guid>https://hitesh.in/2026/local-ai-coding-cost/</guid>
				<description>&lt;p&gt;About a year ago I&amp;rsquo;d have quietly dismissed the idea of replacing Claude with a locally-run model for serious coding work. Then the hardware got more accessible, the models improved, and a large, opinionated group of developers started reporting that they&amp;rsquo;d actually done it - for free - with results good enough to stick with.&lt;/p&gt;&#xA;&lt;p&gt;The catch: &amp;ldquo;free&amp;rdquo; is doing a lot of work in that sentence.&lt;/p&gt;&#xA;&lt;h2 id=&#34;what-developers-are-actually-running&#34;&gt;What developers are actually running&lt;/h2&gt;&#xA;&lt;p&gt;The models getting the most praise right now are Qwen 3.6 35B and Gemma 4 27B, running via llama.cpp or similar. The hardware baseline to run them well is a Mac Studio with 128GB unified memory (&lt;del&gt;$2,000) or a pair of RTX 3090 GPUs (&lt;/del&gt;$4,400 for the pair). One developer in &lt;a href=&#34;https://news.ycombinator.com/item?id=48542100&#34;&gt;a recent thread&lt;/a&gt; reported hitting 150 tokens per second on dual 3090s - fast enough to feel responsive.&lt;/p&gt;</description>
			</item>
	</channel>
</rss>
