I trained a 44M parameter quantized LLM from scratch on 45B tokens. It ships in 19.8 MB and runs at ~1,900 tok/s on CPU. [P]
<!-- SC_OFF --><div class="md"><p>Three weeks back , i posted SHADOW-250M here. It got 360 upvotes, 293 on <a href="/r/LocalLLaMA">r/LocalLLaMA</a> and 94 GitHub stars. Thank you.</p> <p>That model was 60 MB, ran around 400 tok/s on CPU and could retrieve records from an archive on disk. What it couldn’t do reliably was reason over what it retrieved or compute. So I built a smaller one to experiment with those two problems.</p> <p>SHADOW-50M is actually 44M parameters, trained from scratch on 45

