Architecture
Canonical source
This is an overview. The authoritative, maintained design lives in ARCHITECTURE.md at the repo root — it is the source of truth and is kept current under the maintenance contract.
roci's architecture is reverse-engineered from zot's published architecture and adapted to Rust, marking [roci divergence] where it intentionally differs.
Build flavors
roci compiles in two flavors, an architectural invariant:
- minimal — the core distribution API only, with the smallest possible dependency graph.
- full — the core plus optional extensions.
Every extension crate is named roci-ext-* (or roci-cluster) and is reachable from roci-cli only behind a cargo feature, never as an unconditional dependency. just deps-guard (and the minimal-deps-guard CI job) enforces this.
Component & crate model
The workspace separates the stable dist-spec core from cleanly isolated extensions:
roci-core— the OCI Distribution API surface.roci-storage/roci-storage-s3— the storage subsystem (OCI image layout on disk, plus backends).roci-config— the declarative configuration model.roci-telemetry— metrics, traces, and structured errors.roci-ext-sig/roci-ext-search/roci-ext-sync/roci-ext-scan— optional extensions (signatures, search, replication, scanning).roci-cluster— horizontal scale-out.roci-cli— the binary that wires it together behind cargo features.
Storage subsystem
Storage is a plain OCI image layout on disk. Highlights (see the canonical doc for the full design): content-addressable dedup across repositories (reflink → hard link, including on upload), online O(garbage) garbage collection with a grace period and a startup backref rebuild, CRC32C scrubbing with quarantine, per-repository / registry-wide / upload-session quotas, multiple storage paths and an S3-compatible backend behind one Storage trait, an embedded metadata index (append-log + in-RAM maps with compaction, optional HMAC authentication, or an heed/LMDB B-tree KV with volume encryption for at-rest confidentiality) with coalescing write-behind of the spec-visible index.json, and a small-blob LRU content cache that eliminates syscalls on the dominant manifest/HEAD request paths.
Configuration & observability
One TOML file (roci --config) with http (including mTLS fields), auth (htpasswd, ldap, bearer), access_control (IBAC policies), storage (incl. gc, scrub, quota, metadata, subpaths, s3), limits, delete, log, and telemetry sections, validated on load; zero-config defaults need no file. Auth is opt-in; only [access_control] is live-reloadable (2 s poll). The otel build exports traces, metrics, and logs over OTLP and serves a Prometheus scrape view of the same meters; metric labels are bounded by construction, and tail-based sampling is delegated to the OTel Collector. The /metrics endpoint stays unauthenticated.
Scaling
- Vertical — streaming, zero-copy, bounded memory; RSS scales with reference count, not stored bytes.
- Horizontal — clustered instances with repo sharding via consistent hashing (HRW + bounded-load) and a peer proxy.
Kubernetes deployment
A hardened Helm chart (charts/roci/) deploys roci as a single-replica StatefulSet with optional HA S3 storage on RustFS. The chart uses dedicated health endpoints (/readyz, /livez) for Kubernetes probes — unauthenticated and rate-limit-free. In RustFS mode roci creates the bucket itself (create_bucket), and the bucket-init hook confirms every RustFS pod accepts writes. ca_file trusts a private-CA S3 endpoint outside the chart. Each v* release publishes the chart to oci://ghcr.io/jakobmoellerdev/charts/roci, version-locked to the release (chart X.Y.Z deploys image X.Y.Z) and attested. See Kubernetes deployment (Helm chart) in the canonical doc.
Architectural invariants
The canonical doc enumerates invariants that must never regress (build-flavor isolation, extension-crate naming, storage-format stability, and more). Changing an invariant requires updating ARCHITECTURE.md and flagging the change.