Understand the training trade-off among model parameters, data tokens, and compute, from early scaling laws to Chinchilla-style compute-optimal training.
Badges
Badges come from GitHub once the author claims this feed.
Recent posts
every link opens the author’s post-
Scaling Laws From Kaplan to Chinchilla: How Big Should Your Model Be?
-
RoPE: Rotary Position Embeddings and Long Context
Learn how Rotary Position Embeddings rotate query and key pairs so attention scores can use relative token distance.
-
Build a Support-Ticket App with Scikit-LLM
Build a small local support-ticket app that suggests a category and summary while sending uncertain cases to a person.
-
PagedAttention and vLLM: Virtual Memory for the KV Cache
Learn how PagedAttention uses fixed-size KV-cache blocks and block tables to reduce wasted memory when serving many LLM requests.
-
Fine-Tuning with Scikit-LLM: When and How
Learn when Scikit-LLM fine-tuning may help, how to prepare data and why to check provider support before a paid job.
-
Multi-Head Latent Attention: How DeepSeek Compressed the Cache Further
Understand the main idea behind Multi-Head Latent Attention: compact latent storage, projection absorption, and a separate positional key.
-
Scikit-LLM Predictions and Explanations
Separate Scikit-LLM labels from generated explanations and check each statement against the ticket.
-
MHA vs MQA vs GQA: How Modern LLMs Shrink the KV Cache
Compare multi-head, multi-query, and grouped-query attention and see exactly how each changes KV-cache memory.
-
Extract Named Entities with Scikit-LLM
Use Scikit-LLM to mark names, dates and amounts, then check each extracted span against its source.
-
PyTorch SDPA: Shapes, Masks, and Cached Decoding
Use PyTorch scaled dot-product attention correctly and understand its shapes, masks, dropout, and backend choices.
-
Lost in the Middle: Can an LLM Find a Fact Anywhere in a Long 10-K Prompt?
Five hidden 10-K facts moved from the start to the end of 25,000 and 89,000 token prompts with Qwen 3.8: where the model finds them, and where to put the question.
-
Translate Support Tickets with Scikit-LLM
Translate support tickets with Scikit-LLM while checking that names, dates, amounts and the original meaning stay clear.
-
Flash Attention: Why Memory Traffic, Not Compute, Is the Bottleneck
Learn why standard attention wastes GPU memory traffic and how Flash Attention computes the same result in small tiles.
-
Prompt Caching: Put the 10-K First and Prefill Gets 7x Faster
An 11,000-token 10-K prompt in Ollama with Qwen 3.8: how putting the document before the question cut prefill from about 3.5 s to 0.5 s, and how one timestamp broke it.
-
Summarize Support Tickets with Scikit-LLM
Make short support-ticket summaries with Scikit-LLM and check every claim against the source.
-
Prefill vs Decode: Time-to-First-Token and Tokens-per-Second Explained
Understand the two phases of LLM inference, why users notice them differently, and which metrics describe each phase.
-
Structured Output: JSON Mode vs JSON Schema for Reliable LLM Answers
Extract records from 12 real 10-K sentences with Qwen 3.8: a JSON request in the prompt, Ollama's JSON mode and a JSON schema, scored on parsing, shape and correct fields.
-
Dynamic Few-Shot Learning with Scikit-LLM
Choose relevant labeled examples for each new ticket before asking Scikit-LLM for a category.
-
KV Cache Memory Math: How Much VRAM Does Your Context Really Cost
Calculate KV-cache memory from layers, heads, head size, tokens, data type, and batch size.
-
Chain-of-Thought Prompting: Does Step-by-Step Fix LLM Finance Math?
12 margin and growth questions on Amazon and Alphabet 10-K tables with Qwen 3.8: a bare number vs step-by-step vs thinking mode, with accuracy, tokens and time.
-
Build a Scikit-LLM Embedding Pipeline
Join Scikit-LLM embeddings with a local classifier and trace what each step learns.
-
KV Cache Explained: The Trick Behind Fast Generation
Learn exactly what a key-value cache stores, why it speeds up text generation, and why it cannot make decoding parallel.
-
Temperature, Top-k and Top-p: Why the Same Prompt Gives Different Answers
Real next-token probabilities from Qwen 3.8 show how temperature, top-k and top-p pick a word, and why a 10-K fact holds steady while a guess falls apart.
-
Text Embeddings with Scikit-LLM
Turn support messages into numeric vectors and rank related tickets without guessing their similarity.
-
Why Text Generation Is Slow: Autoregressive Decoding Explained
Learn why a decoder writes one token at a time and how the key-value cache avoids repeated attention work.
-
Prompt Injection and Robustness Testing: When the Document Gives the Orders
Eight instructions hidden in a 10-K excerpt vs Qwen 3.8: how often the model obeys them, how a rule, document tags and a reminder cut that to zero, and how to test a prompt.