← Back to the interactive explanation

LoRA Rank and Adapter Anatomy methodology

Version 2026-09-07 · Primary-source ledger: JSON · The three named diagrams are licensed by Kingy.ai under CC BY 4.0.

# LoRA Rank and Adapter Anatomy

Change a layer’s dimensions and adapter rank. Follow the low-rank path and count exactly which parameters you train.

## Run locally

Open `index.html` directly, or run `python3 -m http.server 4190 --bind 127.0.0.1` from this folder and visit http://127.0.0.1:4190/. No package manager, compilation, third-party runtime, account, telemetry, or remote request is needed.

## Model and boundaries

Original LoRA with scaling alpha/r, unmerged adapters, and no bias training. Counts are tensor payloads, not training VRAM or checkpoint file size. They exclude gradients, optimizer states, activations, other layers, and metadata. Rank is not a quality score.

- **forward:** `y = W0*x + (alpha/r)*B*A*x`
- **shapes:** `W0: d×k; A: r×k; B: d×r`
- **parameters:** `base per target=d*k; adapter per target=r*(d+k); multiply by target count`
- **storage:** `adapter bytes=count*r*(d+k)*bytesPerValue; selected base bytes=count*d*k*2`
- **rank_bound:** `rank(B*A) <= min(r,d,k)`
- **break_even:** `adapter has fewer parameters when r < d*k/(d+k)`

### Teaching assumptions

- Every selected target has identical d×k shape.
- FP16 base payload; independently selectable adapter storage precision.
- Original alpha/r scaling, not rank-stabilized LoRA or DoRA.
- Visual matrix blocks are schematic, not area-scaled.
- Changing alpha affects scaling, not parameter count.

## Presets and operation

Select a preset, edit the controls, and inspect the output. Invalid input displays an error and disables exports rather than silently substituting values. Reset restores the selected example. Each preset is a complete state and includes provenance in `presets.json`, dated 2026-09-07. The local `config.js` embeds identical catalog and source data so the page also works from disk; validation checks that it matches the JSON. Update those together with the generation source.

## Primary-source ledger

- **L1: Learn a low-rank update.** The original LoRA paper freezes W0 and learns A and B with an additive BA update, scaled by alpha/r. [research-paper](https://arxiv.org/html/2106.09685). Checked 2026-09-07. Boundary: No universal quality or memory-saving factor is transferred from the paper’s experiments.
- **L2: Implementation variants matter.** PEFT documents rank, target modules, alpha, bias options, initialization, and variants; default A is initialized and B starts at zero. [primary-documentation](https://huggingface.co/docs/peft/en/developer_guides/lora). Checked 2026-09-07. Boundary: This page selects the original formulation rather than modeling every PEFT option.

## Independently reusable assets

| Diagram | Editable source | 2400×1350 raster | Use |
| --- | --- | --- | --- |
| Follow the low-rank path | [SVG](assets/lora-path.svg) | [PNG](assets/png/lora-path.png) | Frozen base plus a trainable branch with explicit shapes. |
| Rank has a storage slope | [SVG](assets/rank-storage.svg) | [PNG](assets/png/rank-storage.png) | A reference parameter and payload ladder. |
| Parameters are not VRAM | [SVG](assets/lora-boundaries.svg) | [PNG](assets/png/lora-boundaries.png) | What the parameter counter includes and leaves out. |

SVGs contain editable text and vector geometry with system-font fallbacks. Kingy may reuse the original assets internally and in its own editorial work; no public redistribution license has been assigned. Source links and attribution policy need to be settled before external distribution. The diagrams are teaching artifacts, not screenshots of hardware or measurements.

## Exports

CSV flattens nested keys into field/value rows. JSON preserves the complete typed object, including topic, schema version, generated time, preset date, inputs, results, formulas, assumptions, and methodology/source paths. Undefined mathematical results are JSON null and CSV `null`. Examples under `exports/` were downloaded from the browser controls and parsed during QA.

## Worked expected result

d=k=4096, r=8, count=64: base per target=16,777,216; adapter per target=65,536; selected base=1,073,741,824; adapter=4,194,304; FP16 adapter=8,388,608 bytes (8 MiB); ratio=0.390625%; alpha/r=2. Small d=2,k=3,r=1 gives 5 adapter versus 6 base values. d=k=8,r=8 gives 128 adapter versus 64 base values; no savings.

## Source map

`index.html` contains the accessible page and explanation; `styles.css` the standalone visual shell; `common.js` input validation and exports; `config.js` the local catalog; `app.js` the topic-specific computation and rendering. `sources.json` and `presets.json` are portable ledgers. `VALIDATION.md` and `output/playwright/` record QA. Authoring source is retained in the companion production kit after season completion.