How far can 12GB of VRAM go? A record of local LLM setup & AI model selection using custom benchmarks

Hello! I'm Takumon, the director of the Takumon IT Research Institute🧪✨I was thinking about what to post next, and I looked into changing the AI model (execution AI) used by my agents🤖💭To make it ...
Hosted on MSN

Local LLM setup: how to use RAG and an embedding model to stop wasting context

Local LLMs are fantastic, and they keep getting better at a staggering pace. I have non-negotiable reasons for preferring a local setup over relying on cloud giants like Claude or ChatGPT. Because of ...
Hosted on MSN

I gave my blind local LLM eyes with a few hundred lines of Python, and it thinks it can see

Many of the open-weight models worth running locally still can't actually see anything, including, up until recently, DeepSeek-V4-Flash-0731. I had the latest DeepSeek V4 Flash model running across ...