What it takes to serve an open weights coding model over an API with Kubernetes, and use it as a coding agent from my laptop, on a home-lab RTX 2060.
Badges
Badges come from GitHub once the author claims this feed.
Recent posts
every link opens the author’s post-
Deploying an Open Weights LLM as an API in Kubernetes
-
Deploying an Open Weights LLM as an API in Kubernetes
What it takes to serve an open weights coding model over an API with Kubernetes, and use it as a coding agent from my laptop, on a home-lab RTX 2060.
-
037_setting_up_a_homelab
Sections
-
Teaching an Agent to Play Space Invaders, Revisited
In 2019 I tried to teach a Deep Q-Network to play Space Invaders and ran out of EC2 credits with an agent barely better than random.
-
Key Value Caching in LLMs
K,V caching mechanism on my whiteboard.
-
Pen and Paper Series: Without non-linear functions, models can't learn complex patterns
Without non-linear functions, neural networks can’t learn complex patterns.
-
Difference between Fine Tuning and RL for LLMs
For the past few weeks I’ve been going through the motions of training an LLM.