Show HN: WiFi-LLM
Hello, while I was learning LLM Inference basics I realized with few tricks I could use my ESP32 for inference if I stream the weights only when they are needed from a host over WiFi. Speed? Really bad. Lol. But this is just a technical demo of how it can be done. I really enjoyed while coding it. It has a sliding window which allows infinite generation ( of course meaningless :) ) Hope you will enjoy! Comments URL: https://news.ycombinator.com/item?id=48992123 Points: 2 # Comments: 0







