01
Efficient AI
Reducing the arithmetic, memory traffic, and storage a model needs before it becomes useful, rather than treating efficiency as an afterthought once training is finished.
Quantization·Compression·Mixed precision·Efficient inference
01 / Research
Central question
How can machine learning systems become more computationally efficient while maintaining useful levels of accuracy and reliability?
Most of the progress in machine learning has come from models that are larger and more expensive. That direction is real, but it is not the only one available, and it is not the one most machines can follow. What interests me is the reverse problem: how much of a model’s cost is actually necessary, and which parts of the cost can be removed without removing what the model is for.
The work below is undergraduate research. It is early, it is done on one consumer GPU, and it is stated as questions and methods rather than as findings.
Fig. 01 — Optimization landscape
02 / Interests
Related, not separate
Eight things I am reading and experimenting my way into.
These are not eight independent interests. They are one question approached from the model side, the mathematics side, the hardware side, and the reliability side.
02
Model Quantization
How low-bit numerical formats reshape a network's error surface — which tensors tolerate coarse grids, which collapse, and why the failure is rarely uniform across a model.
Post-training quantization·INT8 · INT4 · NF4·Calibration·Outliers
03
Hardware-Aware ML
Treating the accelerator as part of the model. Kernel support, memory bandwidth, and tensor-core shapes decide whether a theoretically cheaper model is actually faster.
GPU computing·Memory bandwidth·Kernel support·Benchmarking
04
Mixed-Precision Learning
Allocating precision as a budgeted resource across a network instead of applying one format everywhere, so sensitive components keep the bits they genuinely need.
Layer sensitivity·Precision allocation·Search·Budget constraints
05
Efficient Transformers
Where the cost of transformer inference actually accumulates — attention projections, feed-forward width, KV cache growth — and which of those costs compression can reach.
Attention·KV cache·FFN width·Inference cost
06
ML Systems
The engineering around a model: measurement that is honest, benchmarks that are reproducible, and deployment paths that survive contact with real memory limits.
Inference systems·Profiling·Reproducibility·Deployment
07
Computer Vision
Visual representation learning, and how compression interacts with the features a vision model relies on for fine-grained decisions.
CNNs·Representation learning·Vision transformers·Medical imaging
08
Reliable & Explainable AI
A smaller model still has to be trustworthy. Calibration, uncertainty, attribution, and fairness are part of the efficiency question, not separate from it.
Grad-CAM++·SHAP·Uncertainty·Calibration·Fairness
Aside
A handful of columns can hold most of an attention map’s mass. Those are the values a coarse numerical grid loses first.
This is one of the reasons quantization error is not distributed evenly across a transformer, and one of the reasons uniform low-bit formats fail in specific places rather than degrading gracefully everywhere.
Fig. 02 — Attention field
03 / Current research
Bachelor’s thesis
2026 — Present
Garden City University
Hardware-Aware Mixed-Precision Quantization for Transformers
DevelopingNo results to report yet
Sensitivity to quantization is unevenly distributed across a transformer: some components tolerate four bits, others visibly do not. A uniform format therefore either wastes bits on the tolerant components or damages the sensitive ones. The thesis treats precision as a budget to be allocated, and asks whether allocating it with measured hardware cost inside the loop produces better configurations than allocating from model-side sensitivity alone.
Method
- 01
Transformer
A pretrained model taken as given, with its components treated as separately compressible objects.
- 02
Layer sensitivity
Per-component measurement of how much quality is lost when that component alone is quantized.
- 03
Precision allocation
A bit budget distributed across components according to measured sensitivity rather than uniformly.
- 04
Hardware benchmarking
Execution on a real accelerator, because a format only helps if a kernel exists for it.
- 05
Accuracy · memory · latency
Three numbers reported together, since improving one at the cost of the others is not a result.
- 06
Pareto optimization
Search over allocations, keeping the configurations that nothing else dominates.
- Status
- In development — methodology stage
- Hardware
- RTX 4060 Laptop · CUDA · 8GB VRAM
- Models
- Compact transformer encoders, within an 8 GB budget
- Reported
- Accuracy · P50/P95 latency · peak memory · model size
- Search
- Multi-objective, returning a front rather than a single point
Fig. 03 — Precision map
QKVOFF1FF201040812
- FP16
- INT8
- INT4
Open questions
- Does per-tensor reconstruction error predict end-task loss well enough to drive a search?
- How stable is a sensitivity ranking across calibration samples and measurement choices?
- Do allocation patterns transfer between models, or must they be searched from scratch each time?
04 / Roadmap
Nothing planned is described as done
Completed, in progress, planned — kept distinct.
Completed
2025 — 2026
Explainable Multimodal Medical AI
Skin cancer risk stratification with uncertainty and fairness auditing
Multimodal fusion evaluated at patient level, with attribution, calibration, and demographic audits treated as first-class results.
In Progress
2025 — Present
QuantLab
Hardware-aware mixed-precision quantization framework
A working research framework: sensitivity analysis, precision allocation, and GPU benchmarking in one reproducible loop.
2026 — Present
Hardware-Aware Mixed-Precision Quantization for Transformers
Bachelor's thesis
Extending the QuantLab methodology into a systematic study of how precision should be allocated across transformer components under a fixed hardware budget.
Planned
Next
BitFlex
Adaptive precision transformer inference
Whether precision can be selected per input and per component at inference time rather than fixed once, ahead of deployment.
Later
EdgeFormer
Hardware-aware transformer optimization for constrained devices
Carrying the same allocation question down to devices where memory bandwidth, not compute, is the binding constraint.
Later
ParetoML
Accuracy–efficiency optimization tooling
Making multi-objective model selection a routine, reportable step instead of an ad-hoc comparison of two checkpoints.
Later
AutoQuant
Automated transformer compression
Automating the search over compression decisions so the method transfers to models and accelerators I do not own.