Kingy AI
Subscribe, 2.1M subscribersWork with Kingy
AI News

Scaling Git at OpenAI: Pando Architecture, Performance and Tradeoffs

November 2, 2026 Blog
Git objects, compressed packfiles and shared history, illustrating the concepts behind the Pando guide.

Many agents writing code create a source-control problem as well as a software-engineering problem: safely accepting changes and delivering related history to many readers. Git’s object model, ref updates, and transfer protocol explain the constraints.

This guide separates OpenAI’s report from Git’s documented behavior, examines two public client patches, and explains which scaling controls developers can use with compatible infrastructure today. We have not tested Pando or reproduced OpenAI’s performance results.

What OpenAI says it built

OpenAI’s November 2 report describes Pando, its monorepo Git backend: blob-stored packs, independent ref coordination, regional Express caches, packfile URIs, and hierarchical covers. It claims per-ref serializability and read-your-own-write without global snapshots; O(1) push metadata operations; and O(log Δ) fetch metadata reads, where Δ is the shortest ref-version distance.

OpenAI reports nearly 80% lower median push latency, nearly 1 PB/day, and over 57 TB served in one hour, over 99% of those bytes from cache. It also reports saturating 10-Gbps links. These figures are first-party claims.

Default feature-branch pushes advertise trunk alone, trading repeated off-trunk objects for reusable packs; reported sizes were median <20 KiB and p99 <3 MiB. OpenAI says the Rust backend, built with Codex, reached production in ten weeks and served traffic from April. Validation included conformance tests, deterministic simulations, formal specifications, and agent red-teaming. Its comparison is synthetic, not a production benchmark. The report does not announce public backend availability.

Metadata complexity and end-to-end runtime measure different things. Download, indexing, checkout, and builds need separate measurements.

Why many agents change the shape of Git traffic

A developer opening a branch, pushing a change, and waiting for CI creates several distinct jobs for the source-control service. It must accept objects, check the branch’s current value, update the reference, and supply required history to later readers. The client and CI workers then have their own work: download, verify and index objects, check out files, and run tests.

Consider a deliberately hypothetical workload: 100 agents each push one branch per minute, and four CI jobs fetch each push. That produces 100 pushes and 400 fetches per minute before retries or human activity. Those figures are invented to illustrate fan-out; they are not OpenAI traffic measurements.

One push can therefore create several read demands for related history. A service that handles each request efficiently in isolation can still waste work when it repeatedly assembles similar payloads. Conversely, a cache can only help when requests overlap enough to reuse its contents. Unique changes, cold regions, and constrained client connections alter the result.

The source-control service is also one part of the pipeline. A fast fetch does not reconcile conflicting edits, prove that code works, complete a test suite, or review a change. If all 100 agents target the same branch, they still need to coordinate updates to that branch. Independent branches reduce that particular point of contention, while leaving integration and review to the team.

The Git pieces behind the architecture

Git stores a connected set of objects. A blob holds file content. A tree connects names and file modes to blobs or other trees. A commit identifies a tree and records parent and author/committer information. The object ID is derived from the object’s contents. See the Git object model for the underlying commands and examples.

Imagine a project with src/app.py and docs/guide.md. A developer edits only the application file and commits again. The new commit points to a new root tree and a new application blob. The unchanged documentation subtree can be reused. This is an illustrative repository, not a description of OpenAI’s files.

A ref, such as refs/heads/main, is a named pointer to an object ID. Commits form the history; the branch name tells Git which commit is currently its tip. Updating that name is a small metadata operation even when the reachable history is large. The objects the new tip depends on still have to be available to readers.

Git usually stores and transfers objects in packfiles. A pack can contain compressed object representations and deltas relative to base objects; an index supports lookup. A pack delta is a storage representation and should not be confused automatically with the familiar diff a developer reads in a review. Git supports different repository hash algorithms, so the exact object ID and checksum format depends on the repository. Git pack-format specification

A transfer can omit a base the receiver is known to have. Git calls this a thin pack; it must be made self-contained before normal on-disk use. Git also already has options for reusing existing deltas and object representations. The scaling issue is therefore not that conventional Git always recompresses every object. It is how much work a particular service can share across similar requests, and at what cost in bytes, CPU, storage, and latency. Git pack-objects documentation

A push is a conditional update, not just an upload

A push asks the receiver to move a named ref from an expected old object ID to a proposed new one. If another writer changed the ref first, the expected value no longer matches. The receiver can reject the update instead of silently overwriting that writer’s work. Git’s wire protocol carries the old ID, new ID, and ref name, alongside any required objects. Server-side hooks can also reject an update. Git pack protocol

A local analogue is git update-ref <ref> <new> <old>, which updates the ref only if its current value matches the expected old ID. The same command also has transaction controls for groups of local ref operations. This is useful for understanding the conditional update; it does not disclose Pando’s database schema or establish its behavior for multi-ref pushes. Git update-ref documentation

For an illustrative race, suppose two writers read a branch at commit A. Writer 1 asks to move A to B and succeeds. Writer 2 then asks to move A to C. A conditional check sees that the branch now points to B and rejects writer 2’s stale request. Writer 2 must inspect the new state and decide how to proceed. The object C may exist, but that alone does not make C the branch tip.

This check also explains why rewriting a branch requires care. Git’s explicit --force-with-lease=<ref>:<expected> form makes a forced update conditional on the stated expected value. The shorthand forms rely on remote-tracking refs, which background fetches can change. Git also has --atomic for requesting all-or-none updates of multiple refs, provided the server supports it. Neither feature should be treated as proof of Pando’s support for every push option. Git push documentation

How negotiation and byte delivery can take separate paths

During fetch negotiation, protocol v2 uses want and have messages for desired and existing history. Client and server can also negotiate packfile URIs: HTTP/HTTPS pack locations accompanied by pack-content hashes. The client retrieves those packs before checking connectivity. Git protocol v2 specification

An illustrative request path looks like this:

Git client ── wants / haves ──> Git service
Git client <── pack locations ─ Git service
Git client ── download ───────> HTTP cache or storage
Git client ── verify and index ─> local object store

This illustrates a protocol pattern. A small conversation determines the required history; another route delivers bulk bytes. Reuse has tradeoffs: broadly reusable packs may resend objects, while tailored packs require more per-request work. Costs depend on network transfer, storage, servers, and client processing. The public information does not establish general cost savings.

Capability negotiation also matters. A feature in one build does not establish support in every stable Git release.

A worked example of hierarchical pack coverage

Consider a hypothetical design, with numbers chosen only to make the arithmetic easy. It is not a diagram of Pando’s actual levels, fanout, or fetch algorithm.

Suppose eight successive ref versions each introduce a small pack, numbered 1 through 8. A service could maintain a hierarchy of coverage descriptions: one for each individual pack, one for each adjacent pair, one for each group of four, and one covering all eight. A reader missing versions 1 through 7 could be offered coverage for 1–4, 5–6, and 7. That is three logical groups. It becomes three pack requests only if each selected group is available as one materialized pack; otherwise its underlying packs may still require separate requests. A reader missing just version 8 can use the single-pack entry instead.

The entries are a logical description of which version ranges are covered. They do not, by themselves, say whether the service physically repacks objects into a new file, stores separate prebuilt packs, or assembles a response another way. Physical compaction can create a larger reusable pack, with its own CPU, storage, and cleanup costs. A larger pack can reduce request overhead while including objects a particular client already has; those redundant bytes matter on a slow connection or when clients have limited storage.

In this toy hierarchy, larger ranges can be described using a small number of grouped entries. This is a property of the invented example, not a proof of any production algorithm’s complexity or total running time. Network transfer, pack verification, indexing, and checkout still depend on the bytes and work involved. A production system must also ensure that the selected objects and any delta bases satisfy the receiving client’s requirements.

Client patch 1: separate branch safety from pack prerequisites

OpenAI’s explicit-haves patch separates the current ref value used to check an update from the common history a sender may omit when constructing a pack.

Named ref IDs remain available for matching and old-ID checks, including force-with-lease. They no longer automatically become pack prerequisites. Instead, .have pseudo-refs and explicitly acknowledged common objects establish those prerequisites. A server relying on this behavior must check that the client requested the capability.

Here is an invented example. A client knows a branch points to B and proposes C. The receiver uses B to check that the branch has not moved. Separately, the parties agree that only objects through A can be omitted. That distinction lets a replacement payload avoid depending on objects that disappear with the pack it replaces, while retaining the branch’s expected-old-value check.

A ref tip answers a concurrency question; a pack prerequisite answers a data-availability question. The public patch documents this mechanism, without establishing a measured production speedup.

Client patch 2: parallel downloads need compatible dependencies

Multiple pack URLs do not guarantee that a client can install every pack independently. A later pack can contain a REF_DELTA whose base comes from a pack processed earlier. If installation depends on that order, opening several network connections alone will not remove the dependency.

OpenAI’s parallel URI indexing patch addresses this by negotiating no-ref-delta for the relevant packs. It allows concurrent processing under that promise and verifies packs using index-pack --no-ref-delta. A pack that violates the promise fails validation. The fetch.packfileUriJobs setting limits the number of downloads and indexers; it defaults to one, and higher values take effect only with the negotiated promise. A single URI or job remains serial.

This is a useful example of client/server co-design. Parallelism needs both available work and a dependency contract that permits that work to run safely at the same time. More jobs can also compete for CPU, disk, and bandwidth, so the right limit depends on the client and the workload. The source documents the change; it does not establish how much faster any particular repository will fetch.

Consistency: one branch, several branches, and object availability

Distributed-system descriptions often compress several guarantees into the word consistent. For a Git user, it helps to ask which observation is being guaranteed.

Property Practical meaning
Conditional update of one ref A writer can ask to change a branch only if it still has the expected old value.
Ordering for one ref Updates to that branch have a coherent order under the service’s stated contract.
Atomic multi-ref update A group of ref changes succeeds or fails together when the server supports the requested operation.
Repository-wide snapshot A reader sees multiple refs as they existed at one common point in time.
Object availability The history advertised to a reader must be usable under the protocol and service contract.

These are separate properties. In an analytical example, suppose branch alpha advances at 10:00:01 and beta advances at 10:00:02. A reader observing both without a repository-wide snapshot might see an older value of one and a newer value of the other. That observation alone does not mean either branch was corrupted. It does mean software that depends on a coordinated relationship between refs must understand the service’s actual contract.

Likewise, per-ref ordering does not by itself establish whether a server supports an atomic push affecting several refs. The public material reviewed for this guide does not establish Pando’s full multi-ref atomic contract. Teams with release tooling that updates several refs together should verify the exact behavior with their own host rather than infer it from a single consistency label.

How to read the performance figures

A median describes the middle observation in a distribution. It does not tell you the slowest requests, the 99th percentile, or the experience of every repository. A percentage reduction also needs a baseline: as original arithmetic, an exact 80% reduction would leave 20% of the original latency, making the before value five times the after value. An approximate percentage reduction yields an approximate ratio.

Bandwidth figures need units and scope. At an ideal 10 gigabits per second, dividing by eight yields 1.25 gigabytes per second before protocol, TLS, and other overhead. That is raw transfer arithmetic, not a measurement of checkout or build completion. An hourly figure for an entire service is aggregate traffic; it should not be compared as though it were the throughput of one laptop. One busy hour does not establish a sustained daily rate.

A cache-hit figure measured by bytes answers a different question from a hit rate measured by requests. A few large successful cache deliveries can dominate the byte total. To evaluate a system for your own workload, ask for request counts, payload sizes, cold and warm behavior, geographic distribution, latency percentiles, and failures alongside throughput. Simulations and formal specifications do not establish that an implementation is bug-free or substitute for independent benchmarks.

What is public, and what is upstream

OpenAI’s Git fork is public source code. It is a client/build artifact; cloning it does not install the Pando backend or grant access to OpenAI’s infrastructure.

The public patch tracker distinguishes changes carried in OpenAI’s build, proposed upstream work, and upstream integration status. It lists the two patches discussed above in the OpenAI build. A proposal, an integration branch such as seen, and a stable Git release are different states. Check the tracker and the release you actually run before relying on a particular change.

Two narrower upstream examples show why production-oriented Git work includes reliability and observability. One Git commit handles a resumed pack download where the staging file already has all bytes and the repeat HTTP Range request returns 416. Git can then run index-pack to validate the existing file; this is not a blanket instruction to ignore download errors. Another Git commit adds Trace2 instrumentation for URI-pack download time and the number of advertised packs. That improves visibility into a transfer phase; instrumentation alone is not a speedup.

Git scaling tools available today

Most teams do not need a new backend to investigate a slow clone. Start by measuring whether the delay comes from remote negotiation, transferred bytes, local indexing, checkout, or later build steps. Then choose controls for the part causing trouble. The following are standard Git features, subject to your client version and host’s support; they are not Pando switches.

Control What it changes Tradeoff to check
Partial clone with --filter=blob:none Defers file-content blobs until they are needed. Missing content can require later network requests.
--sparse and sparse checkout Limits the initial set of files placed in the working tree. It addresses checkout scope, not necessarily the historical objects already transferred.
Shallow clone with --depth Limits the history initially retrieved. Jobs that need older ancestors may fail or need more history.
--bundle-uri Seeds a clone from a bundle before fetching from the remote. Check host support and the documented incompatibility with specified shallow options.

The Git clone manual documents those options and their constraints. Partial clone and sparse checkout can be combined when both transfer and working-tree size matter, but they solve different problems. Test with the actual CI job: a smaller initial clone that triggers repeated missing-object fetches later may move delay rather than remove it.

For sparse checkout, Git recommends cone mode. It selects directories while including certain files in ancestor locations; merge or rebase can materialize files outside the selection. A sparse index can reduce index work for some commands, but Git’s documentation still marks it experimental and warns about compatibility with external tools. Git sparse-checkout documentation

For a team running many agents, a useful first investigation is to record one representative push and its downstream fetches. Count how many jobs retrieve related history, distinguish cold from warm runs, and measure transfer separately from checkout and test execution. That evidence identifies whether shared delivery, smaller working trees, fewer redundant jobs, or branch coordination deserves attention first.