03 / Notes
Working notes.
Things I am trying to understand, written down while I am still unsure about them. These are notebook entries rather than publications: each one records what I have read, what I have observed, and what I cannot yet answer.
6 notes · 2026
06 / 18 July 2026
6 min
What makes INT4 quantization difficult
Sixteen reconstruction levels is not many. A look at why the step from INT8 to INT4 behaves less like a smaller version of the same problem and more like a different one.
Quantization · INT4 · Outliers
05 / 30 June 2026
5 min
Why some transformer layers resist quantization
Sensitivity is not uniform across depth or across components. What I keep observing, and which explanations I find convincing so far.
Sensitivity · Transformers · Mixed precision
04 / 11 June 2026
6 min
Accuracy, latency, memory: reading a three-way trade-off
A single number cannot describe a compressed model. Notes on Pareto fronts, dominated configurations, and why reporting one axis at a time is misleading.
Pareto · Evaluation · Multi-objective
03 / 24 May 2026
5 min
Where VRAM actually goes during inference
Weights are the part everyone counts. On an 8 GB laptop GPU, the parts nobody counts are what decide whether a model runs at all.
GPU · Memory · Inference
02 / 2 May 2026
4 min
How much calibration data does post-training quantization need?
Post-training quantization asks for a handful of examples and then makes decisions that affect every input. What that handful has to represent.
PTQ · Calibration · Methodology
01 / 15 April 2026
5 min
Why mixed precision, rather than one format everywhere
If sensitivity varies across a model, a uniform format is a budget spent badly. The argument, and the cost of acting on it.
Mixed precision · Allocation · Hardware