Astra Field Guide: Inspect and Reproduce the Tests

Start by deciding whether you want to inspect a retained artifact or create a new one. Inspecting saved evidence can confirm what the record contains. A new run produces new evidence and needs its own results, even when its prompt matches the original.

The observed conditions

Both pairs were individual, time-separated case studies. Authoring included self-checking and changes before the first completion claim. The app audit and research audit document the retained provenance and material corrections; the comparison articles provide the final review method and findings.

The guide uses four labels: observed for execution evidence, saved-documentation for captured official-source claims, untested for a procedure without a completed local trial, and planned for the unsubmitted motion demonstration. Saved source pages do not establish today’s availability, prices, or product behavior.

Preserve prompt and input identity

Use the plain-text files when exact bytes matter:

From the delivery directory, this read-only command lets you compare your copies with those values:

shasum -a 256 prompts/working-app-exact.txt prompts/research-spreadsheet-exact.txt

For a new matched app test, copy the two SignalDesk inputs into each fresh run directory under the filenames the prompt expects. For research, copy research-workbook-brief.md. Keep completed candidate artifacts and the other model’s output out of the authoring context. Record input hashes, settings, runtime/tool versions, permissions, and the actual prompt submission method.

Record start and completion times with their timezone and stated boundaries. Keep setup, authoring, correction, and operator verification intervals separate. The app’s recorded time to first claimed completion is based on metadata; its retained CLI events lack per-event timestamps. Timing and account-wide allowance changes do not establish per-run cost or a general speed ranking.

Inspect the retained apps

The following commands are reader procedures supplied with this guide; they were not a new app test during editorial preparation. They assume Python 3 is already installed and that your shell is in the delivery directory. Run each server in a separate terminal:

python3 -m http.server 8765 --bind 127.0.0.1 --directory artifacts/apps/astra

python3 -m http.server 8766 --bind 127.0.0.1 --directory artifacts/apps/sol

Open http://127.0.0.1:8765/ and http://127.0.0.1:8766/ in your browser. Separate ports keep the apps’ localStorage state on separate origins. Use clean browser profiles or clear only the storage for these local origins before a fresh checklist. Stop each server with Control-C when finished. The example.test source URLs are fictional and should remain inert.

The working-app checks

These are the fixed checks behind the recorded 12/12 results:

The retained scripts performed browser actions, including typing, selection, keyboard use, reload, and viewport changes. The suite does not independently cover every summary count, all filter combinations, or full focus trapping. A 12/12 result is narrower than a production or accessibility audit.

The historical author-run desktop stills have different dimensions and should not be described as matched 1440×900 captures. The delivery also includes new matched Astra and Sol desktop captures from the replacement review; distinguish those from the historical stills. The selected historical mobile stills are both 390×844.

Sol’s Playwright wrapper made failed npm-registry lookups in three command executions; no download or installation completed. That attempt belongs in the scope record even though the independent browser suite observed only loopback traffic.

Inspect the workbooks and their sources

Use copies of the Astra XLSX and Sol XLSX for any interactive changes. The source index identifies the retained official-page snapshots. Preserve their dates and original source URLs.

The historical prompt asked for current documentation. Replaying it with fresh retrieval can change the evidence set. Replacing retrieval with sources/ is an untested offline adaptation, provided in the beginner guide. It can support claims about the supplied snapshots; it cannot be described as a byte-identical repeat of the historical task.

The fixed research checks were:

For formula verification, trace the four workload types to their rate cells and source conditions. Change one input, then remove one required rate in memory or a disposable copy. Check that recalculation changes the affected result and that a missing rate suppresses that model’s output with a visible explanation. Verify the preserved workbook’s bytes remain unchanged.

Observed historical evidence includes runtime imports and in-memory mutations of the saved XLSX files, passing calculator assertions, and preserved file hashes. It supports runtime recalculation. A later in-memory review found a remaining Sol limitation: removing a required base rate blanks the affected total but does not dynamically name the missing rate. Astra displays “Missing API rate.” The recorded final 12/12 totals remain historical results; they do not certify that this later diagnostic condition passed. No candidate repair was made. Sol’s permitted correction had already been used.

Native Excel interaction was not tested. An Excel or LibreOffice session would be a separate compatibility check and should be described by the application and actions actually used.

Read the research correction accurately

Astra’s initial and final results are 12/12. Sol’s initial record says 10/12: #7 failed for a 2× cache multiplier unsupported by its cited source set, and #11 lacked sufficient visual proof at an auto-crop boundary. Check #12 nevertheless passed despite also prohibiting unsupported values. Preserve the original total with that audit caveat rather than silently changing it.

The wider saved source record includes an official pricing page that supports the rates. Source coverage is an observed difference; source drift is a possible limitation, not an established cause. Sol’s initial adjusted estimates were already blank. Its correction revised source labels and separated input/cache multipliers; it did not newly blank a previously populated estimate.

Sol received the generic acceptance-audit correction prompt printed in the beginner guide. An operator later produced an explicit A1:F37 render without editing the candidate. Keep that proof render separate from the author’s correction. The recorded final fixed checks were 12/12. Sol’s original first-completion XLSX was not separately retained; initial behavior claims rely on the preserved inspection records and initial hash.

Keep review and provenance claims bounded

Functional verification and final-artifact preference answer different questions. For the review findings and the actual review method, use the working-app comparison and research comparison. Historical preference scores are superseded and are not reproduced in this guide.

For a new blind review, use a reviewer who has not seen model identities or the identified originals. Prepare neutral copies, randomize order, withhold the mapping until scoring is complete, and retain the review record. State whether the reviewer interacted with the artifact, inspected renders, or both. Neutral filenames alone do not establish reviewer isolation.

The original recordings have limitations. Astra’s app run lacks literal selector-screen recording; Sol’s app run shows an operator runner banner rather than a native selector. Research launch records establish worker settings; captured GUI selectors belong to the operator context. Raw recordings contain unrelated foreground content and remain private. A screenshot, banner, or rendered workbook should be described as the evidence it actually is.

Motion has no reproduction result yet

Planned. The motion protocol freezes the brief, exact prompt, checks, correction policy, and recording requirements. The readiness record found existing local executables; the preflight result explains why submission stopped when the required visible run evidence could not be recorded. Neither model authored a candidate.

A future run must establish that recording prerequisite and then prove the complete offline composition/render path. Version probes alone do not prove rendering. The planned output uses HyperFrames and FFmpeg, so any future comparison must attribute those tools. No Ultra test or native media-generation result exists in this delivery.