Quantization Lab methodology

Version 2026-09-07 · Primary-source ledger: JSON · The three named diagrams are licensed by Kingy.ai under CC BY 4.0.

# Quantization Lab

Move eight values onto an integer grid. See every rounded code, clipped value, and reconstructed number.

## Run locally

Open `index.html` directly, or run `python3 -m http.server 4190 --bind 127.0.0.1` from this folder and visit http://127.0.0.1:4190/. No package manager, compilation, third-party runtime, account, telemetry, or remote request is needed.

## Model and boundaries

This is per-tensor linear quantization of a tiny vector. It does not model GGUF blocks, NF4, optimizer memory, model quality, or packed-file overhead beyond the stated metadata.

- **symmetric:** `qmax=2^(b-1)-1; qmin=-qmax; s=max(|calMin|,|calMax|)/qmax; z=0`
- **affine:** `lo=min(calMin,0); hi=max(calMax,0); qmin=-2^(b-1); qmax=2^(b-1)-1; s=(hi-lo)/(qmax-qmin); z=clip(roundEven(qmin-lo/s))`
- **quantize:** `q=clip(roundEven(x_fp16/s)+z,qmin,qmax); reconstructed=(q-z)*s`
- **storage:** `FP16 baseline=2N bytes; packed integer payload=ceil(N*b/8); illustrated metadata=4-byte scale + 1-byte zero point`
- **error:** `MAE=mean(abs(reconstructed-x_fp16)); clipping counted when rounded integer falls outside range`

### Teaching assumptions

- One FP32 scale and one signed INT8 zero point per vector, including symmetric mode.
- Narrow symmetric INT4 uses -7…7, unlike full signed INT4 -8…7.
- FP16 source conversion is simulated; scale and reconstruction arithmetic use JavaScript numbers.
- No quality or speed inference follows from the numerical error.

## Presets and operation

Select a preset, edit the controls, and inspect the output. Invalid input displays an error and disables exports rather than silently substituting values. Reset restores the selected example. Each preset is a complete state and includes provenance in `presets.json`, dated 2026-09-07. The local `config.js` embeds identical catalog and source data so the page also works from disk; validation checks that it matches the JSON. Update those together with the generation source.

## Primary-source ledger

- **Q3: FP16 source format.** OpenEXR documents half precision with one sign bit, five exponent bits, ten fraction bits, and subnormal values. [primary-documentation](https://openexr.com/en/latest/TechnicalIntroduction.html#the-half-data-type). Checked 2026-09-07. Boundary: Only finite values up to 65504 are accepted here; conversion uses nearest-even rounding as an explicit policy.
- **Q1: Round, shift, saturate.** ONNX QuantizeLinear specifies scale, zero point, saturation, and nearest-even rounding. Signed INT4 spans -8 through 7. [primary-documentation](https://onnx.ai/onnx/operators/onnx__QuantizeLinear.html). Checked 2026-09-07. Boundary: The symmetric teaching scheme deliberately leaves the most negative code unused.
- **Q2: Reconstruct through the scale.** ONNX DequantizeLinear defines reconstruction as (integer − zero point) × scale. [primary-documentation](https://onnx.ai/onnx/operators/onnx__DequantizeLinear.html). Checked 2026-09-07. Boundary: Floating-point implementations may introduce additional rounding.

## Independently reusable assets

| Diagram | Editable source | 2400×1350 raster | Use |
| --- | --- | --- | --- |
| The integer grid | [SVG](assets/integer-grid.svg) | [PNG](assets/png/integer-grid.png) | Scale and zero point locate representable values. |
| Rounding versus clipping | [SVG](assets/clipping-error.svg) | [PNG](assets/png/clipping-error.png) | Two distinct ways a reconstructed value can differ. |
| Where the bytes go | [SVG](assets/storage-ledger.svg) | [PNG](assets/png/storage-ledger.png) | Payload savings with metadata made visible. |

SVGs contain editable text and vector geometry with system-font fallbacks. Kingy may reuse the original assets internally and in its own editorial work; no public redistribution license has been assigned. Source links and attribution policy need to be settled before external distribution. The diagrams are teaching artifacts, not screenshots of hardware or measurements.

## Exports

CSV flattens nested keys into field/value rows. JSON preserves the complete typed object, including topic, schema version, generated time, preset date, inputs, results, formulas, assumptions, and methodology/source paths. Undefined mathematical results are JSON null and CSV `null`. Examples under `exports/` were downloaded from the browser controls and parsed during QA.

## Worked expected result

Eight balanced values at symmetric INT4, scale 3/7, give codes [-7,-2,-1,0,1,2,5,7]. MAE is 1/14 = 0.0714285714. Payload is 4 bytes; total including 5-byte metadata is 9 bytes versus 16 FP16 bytes. The clipping preset saturates 2 values. With scale 1, ties -2.5,-1.5,-0.5,0.5,1.5,2.5 map to -2,-2,0,0,2,2.

## Source map

`index.html` contains the accessible page and explanation; `styles.css` the standalone visual shell; `common.js` input validation and exports; `config.js` the local catalog; `app.js` the topic-specific computation and rendering. `sources.json` and `presets.json` are portable ledgers. `VALIDATION.md` and `output/playwright/` record QA. Authoring source is retained in the companion production kit after season completion.