Design edition 0.1

Anthera Data Plane

One kernel, explicit authorities, bounded at every edge.

Fig. A — The stable reading trace. Runtime topology and narrative order are related, not identical.

Source: fe342ee
Status: Implementation-ready proposal; runtime not deployed

route

The system thesis

Nautilus owns the trading kernel; Anthera extends one canonical trace into durable history and multi-user delivery.

Question: What must remain true from ticker selection to chart paint?
Binding decision: The browser reaches data only through an authenticated application gateway; every queue is bounded and recoverable.
Owner input: Production SLOs, provider budgets, and data-rights limits remain owner decisions.

Fig. A — The stable reading trace. Runtime topology and narrative order are related, not identical.

Reading focusThe browser reaches data only through an authenticated application gateway; every queue is bounded and recoverable.

Verify the design packagemise x -- task verify

Prompt structure, Rust harness, lint, tests, and state checks pass.

The trace is the product boundary

The target begins with Databento decoded directly into canonical Nautilus values. LiveNode routes those values through DataEngine, Cache, MessageBus, strategies, portfolio, risk, and execution. Anthera extends that proven kernel into QuestDB history, rebuildable Redis projections, and an authenticated browser gateway without inventing a second market model.

Live updates use append-or-replace semantics instead of rebuilding the visible window.
Historical reads use opaque cursors against a stable committed watermark.
Cold symbols attach live ownership while bounded history work begins in parallel.
A reconnect either replays a complete suffix or emits an explicit resync instruction.

EvidenceOfficial Nautilus architecture and Databento integration docs; frozen Rust source; accepted Nautilus-first ADR.

Authority is explicit

Nautilus owns in-process market and trading semantics. Its event store owns state-affecting run history. QuestDB owns bulk hot Quote, Trade, and Bar history. Redis accelerates service reads and coordination but remains rebuildable. The Cache database is an engine-state projection, not the browser's infinite-history authority.

Durable acknowledgement follows the declared QuestDB commit boundary.
Provisional delivery is labeled and always has a committed recovery path.
Provider, dataset, schema, Nautilus identity, generation, and provenance cross every service boundary.

EvidenceOfficial Cache and Event Sourcing docs plus the frozen persistence and infrastructure source audit.

split

Current to target

Preserve proven contracts, rewrite coordination bottlenecks, and delete nothing until a compatible replacement owns the behavior.

Question: What changes, and what evidence must exist before it changes?
Binding decision: Phase 0 classifies every current module and provisional behavior as preserve, intentionally change, or delete.
Owner input: The target Tradingview repository must confirm every current-state hypothesis from code and runtime evidence.

Preserve when proven

Owned ports
Wire fixtures
Cursor and generation
Correction behavior

Rewrite behind ports

Per-viewer replay
Unbounded watchers
Per-page senders
Single-replica ownership

Delete after rollback

Superseded flags
Dead coordination
Duplicate schemas
Expired compatibility paths

Planned target gate · Freeze current contractsmise x -- task contracts:test

Acceptance target: Characterization fixtures prove the accepted current behavior.

Provisional preserve candidates

Typed Rust and TypeScript contracts, provider ports, page identity, eligibility rules, partial-page behavior, cursor recovery, and bidirectional paging are candidates—not facts. Characterization tests must freeze the real behavior before architecture work depends on it.

Keep owned ports even when adapters are replaced.
Freeze generation and cursor semantics in cross-language fixtures.
Preserve correction and exhaustion behavior only when executable tests confirm it.

EvidenceExecPlan: Provisional preserve candidates and Phase 0.

Rewrite candidates

The audit must specifically test for synchronous Redis replay per viewer, process-global watcher growth, per-page QuestDB senders, table-wide WAL waits, source-limit truncation, full-window frontend sorting, and single-replica ownership presented as scalable.

A rewrite names the preserved port and compatibility test.
Shadow reads and writes precede client cutover.
Old paths remain independently switchable until rollback evidence passes.

EvidencePrompt: current-system audit hypotheses.

contract

Owned boundaries

Nautilus owns market and trading meaning; Anthera owns service metadata, durable projections, and browser contracts.

Question: Which types are allowed to become durable contracts?
Binding decision: Nautilus model values are canonical inside the runtime; infrastructure and browser shapes remain narrow projections.
Owner input: Exact target crate paths are reconciled against the Tradingview workspace in Phase 0.

Owned envelopeOwned boundaries

source: provider · dataset · schema · instrument
series: aggregation · step · price source · generation
time: event · receive · commit watermark
order: record identity · sequence · correction lineage
scope: tenant · entitlement · query budget
proof: coverage · completeness · exhaustion · provenance

External SDK and infrastructure types terminate outside this boundary.

Enforce crate directionmise x -- task check:arch

Outward dependencies and missing workspace lint inheritance fail.

The kernel dependency rule

The runtime composes core, model, live/node, Databento, trading, and IBKR at one revision. Data, system, portfolio, risk, and execution form the coherent kernel. QuestDB, Redis, workflow, and browser adapters depend outward from bounded Anthera projection ports.

Nautilus model → kernel → bounded projection ports
QuestDB, Redis, workflow, and browser adapters implement Anthera service ports
runtime and CLI own lifecycle, composition, and proof commands
every workspace crate inherits strict Rust lints

EvidenceARCHITECTURE.md and scripts/check-arch.sh.

The provider is already integrated

nautilus-databento uses Databento's official Rust client and supplies file loading, historical requests, live subscriptions, a LiveMarketDataClient, DBN decoding, and Arrow/catalog support. Anthera validates that adapter instead of building another normalization path.

DBN schema coverage, symbology, timestamps, precision, and skipped-record behavior are fixture tested.
All Nautilus crates share one revision, feature set, precision policy, and license review.
Raw DBN and broad internal Data enums do not become QuestDB or browser contracts.

EvidenceOfficial Databento integration docs and crates/adapters/databento source.

contract

Identity and time

A record is not identified by timestamp alone; source, series, generation, correction lineage, and durability travel with it.

Question: How can retries, corrections, and aggregation remain deterministic?
Binding decision: Nautilus types carry market identity and event/init time; Anthera adds only source, durability, coverage, and entitlement metadata.
Owner input: Final provider record keys and correction guarantees require DBN fixture evidence.

Owned envelopeIdentity and time

source: provider · dataset · schema · instrument
series: aggregation · step · price source · generation
time: event · receive · commit watermark
order: record identity · sequence · correction lineage
scope: tenant · entitlement · query budget
proof: coverage · completeness · exhaustion · provenance

External SDK and infrastructure types terminate outside this boundary.

Planned target gate · Exercise schema boundariesmise x -- task schema:test

Acceptance target: Round trips, malformed inputs, and generation races are covered.

Canonical value plus service envelope

The payload is a Nautilus QuoteTick, TradeTick, Bar, order-book value, instrument, or adapter-specific supported type. Anthera's envelope adds provider dataset and DBN schema, raw-source identity, entitlement scope, generation, correction lineage, coverage, commit watermark, and provenance without redefining the payload.

Event identity deduplicates provider retries without erasing valid corrections.
Series identity includes aggregation, step, price source, schema version, and generation.
QuestDB, Redis, and browser keys derive from stable Nautilus identity plus explicit service metadata.

EvidenceOfficial DBN mapping and timestamp docs plus the Nautilus model/data audit.

Watermarks state what is safe

A provisional live sequence tells the viewer what it has seen. A committed watermark tells readers what QuestDB has acknowledged. Coverage intervals describe which historical ranges are complete, while generation prevents a stale response from contaminating a newer symbol or timeframe.

A cursor binds to a stable watermark and semantic series identity.
Completeness and exhaustion are explicit response fields.
Late events and corrections advance lineage rather than silently rewriting client state.

EvidencePrompt: durability and query contracts.

temporal

The live path

One upstream subscription serves many viewers through fenced ownership, bounded fanout, and typed recovery.

Question: How does low latency coexist with durable and gap-free semantics?
Binding decision: Every viewer queue is bounded; owner death and suffix loss produce deterministic replay or resync.
Owner input: WebSocket versus SSE and the internal broker are selected only from measured control, proxy, and recovery behavior.

01Authorizerequest
02Snapshotbounded transition
03Attach ownerbounded transition
04Replay suffixbounded transition
05Live tailbounded transition
06Typed resyncclient state

Planned target gate · Prove live recoverymise x -- task fanout:chaos

Acceptance target: Owner death, partition, reconnect storm, and slow-client cases remain bounded.

Attach without a gap

The gateway authorizes the series, obtains a stable snapshot and committed watermark, attaches to the shared live owner, replays the suffix after that watermark, and then hands the viewer to the live tail. The snapshot-plus-replay handshake closes the race between history and subscription.

One source event is recorded once, never once per viewer.
Sequence and generation accompany every snapshot, replay, and live frame.
Heartbeat, lag, gap, resync, cancellation, and close are typed control messages.

EvidencePrompt: stream contract and ExecPlan Phase 5.

Ownership is fenced

One Nautilus data-client subscription feeds the kernel once. A separate bounded projection seam writes canonical values to QuestDB and publishes service events for many viewers. Multi-replica ownership uses leases and fencing outside the single-node kernel; stale owners cannot commit or publish after ownership changes.

Nautilus external MessageBus egress is lossy by contract and cannot be the authoritative writer.
Exactly one multi-replica broker role is selected.
Slow consumers shed, resync, or disconnect before they exhaust process memory.

EvidenceExecPlan Phase 5 and framework evaluation contract.

temporal reverse

Cold ticker and infinite history

A cold selection becomes useful immediately while one durable, coalesced workflow fills missing history behind an opaque cursor.

Question: What happens when the requested series does not exist locally yet?
Binding decision: Interactive demand is progressive, restart-safe, coalesced, authorized, and limited by provider budget.
Owner input: The exact interactive versus repair priority weights require workload and cost evidence.

01Viewport demandrequest
02Coverage querybounded transition
03Coalesce rangebounded transition
04Fetch DBNbounded transition
05Commit batchbounded transition
06Prepend barsclient state

Planned target gate · Crash every workflow edgemise x -- task backfill:crash-matrix

Acceptance target: Retries produce one logical history and deterministic cancellation.

First useful data wins

On ticker selection, the gateway authorizes demand, queries committed coverage, attaches live ownership, and joins or starts bounded history work. Existing committed chunks stream first. New pages become visible only after their QuestDB batch watermark advances, then merge with the live tail without duplicates.

Visible-range demand is exclusive and cancelable.
Opaque cursors prevent clients from manufacturing storage offsets.
Progress and exhaustion guards prevent fetch loops while scrolling left.

EvidencePrompt: cold ticker and frontend rules.

Workflow granularity is a range

Restate, if selected over a small owned job table, coordinates one bounded provider/dataset/schema/instrument/range/generation workflow. It authorizes, fences, pages through the Nautilus Databento historical client, receives canonical Nautilus values, commits them, advances coverage, publishes, and reports terminal state. It never runs once per market tick.

Overlapping requests coalesce by workflow key.
Kill-point tests cover every external side effect.
Watchlist warming and bulk repair cannot starve active-ticker demand.

EvidenceExecPlan Phase 4.

decision

Storage and coordination

The Cache database, event store, QuestDB catalog, Redis projections, and cold files each have one explicit authority.

Question: Which state belongs in which system, and how is it recovered?
Binding decision: No persistence mechanism silently substitutes for another; Anthera Redis loss is correctness-neutral and QuestDB commits are explicit.
Owner input: Partition size, retention, cache byte budget, and cold-tier activation depend on measured workload and cost.

Scroll horizontally to inspect the full decision ledger.

Decision ledger for Storage and coordination
Candidate	Decision	Owned role	Activation evidence
QuestDB	Adopt	Authoritative bulk hot market history	Exact Nautilus-value round trip and outage matrix
Nautilus event store	Conditional	State-affecting run history and replay order	Crash, snapshot, verification, and live-recovery proof
Nautilus Cache database	Conditional	Selected engine-state restart projection	Prove supported types and rebuild semantics
Anthera Redis	Adopt	Rebuildable service projection and coordination	Deletion must be correctness-neutral
Parquet / Vortex	Later	Catalog and immutable cold history	Activate from measured retention and query evidence

Planned target gate · Compare storage truthmise x -- task storage:shadow-diff

Acceptance target: Coverage, counts, hashes, and watermarks match frozen fixtures.

Three durable histories

QuestDB stores bulk Quote, Trade, and Bar history. The event store records ordered state-affecting history and joins market context through ReplayCatalog. The optional Cache database backs selected engine projections. These roles never collapse into one database.

A bounded QuestDB projection writer exposes batch visibility and continuous coverage.
Event-store sequence remains the authority for trading-state replay order.
Cache capacity and cache-database support do not define infinite market history.

EvidenceOfficial Cache and Event Sourcing docs; frozen persistence, event-store, and infrastructure source.

Service projections remain disposable

Anthera Redis stores versioned pages, current values, coalescing, leases, fencing, and bounded shared replay. It is distinct from Nautilus's cache adapter and can be rebuilt from QuestDB. Parquet is the proven Nautilus catalog/cold format; Vortex competes only after a measured trigger.

No XRANGE occurs for every delivered viewer event.
Follower waits and replay retention are bounded.
Vortex competes with Parquet only after a retention-cost trigger.

EvidenceExecPlan Phases 3 and 7.

decision

Framework and Nautilus decisions

Leverage means adopting the coherent Nautilus kernel, then owning only the distributed service gaps it leaves open.

Question: Which dependencies buy durable leverage instead of operational overlap?
Binding decision: Nautilus is the trading runtime; every additional framework must solve a measured gap outside that kernel.
Owner input: All dependency versions and licenses are rechecked immediately before adoption in the target repo.

Scroll horizontally to inspect the full decision ledger.

Decision ledger for Framework and Nautilus decisions
Candidate	Decision	Owned role	Activation evidence
Nautilus kernel	Adopt	Canonical trading runtime	One revision, feature, precision, and license policy
nautilus-databento	Adopt	Historical and live Databento data client	DBN fixture and failure proof
nautilus-interactive-brokers	Adopt	Live execution client	Paper execution and reconciliation proof
nautilus-event-store	Conditional	State-history and replay	Crash and recovery proof
iii	Conditional	Plugin/adapter experiment	Only for a demonstrated execution-model need
Supabase ETL	Reject	No hot-path role	Reopen only for separate PostgreSQL CDC
iroh	Reject	No browser role	Reopen for measured node-to-node transfer
warpgate	Operations	Staff bastion only	Never a data-plane dependency

Inspect dependency policymise x -- task security

Advisories, licenses, bans, and sources satisfy the current policy.

Nautilus is the runtime

LiveNode composes model, DataEngine, Cache, MessageBus, Trader, Portfolio, RiskEngine, and ExecutionEngine. nautilus-databento produces canonical values through the official SDK; IBKR completes live execution. Adopt the system compositionally, not as unrelated crates.

Direct runtime: core, model, live/node, trading when used, Databento, and IBKR.
Kernel closure: common, data, system, execution, portfolio, risk, analysis, network, and cryptography.
Conditional: event-store/live; persistence plus serialization/arrow for catalog or cold history.
Development or deferred: backtest, selective testkit, indicators, plugin, PyO3, Nautilus CLI, cloud, and PostgreSQL infrastructure.

EvidenceOfficial architecture, Databento, Cache, Event Sourcing, Live, and IBKR docs plus frozen crate source.

Framework roles stay narrow

iii, Restate, Supabase ETL, iroh, Warpgate, and additional brokers remain outside the kernel until a measured unsolved requirement names their role. Arrow/DataFusion and Parquet are already part of Nautilus catalog capabilities; Vortex remains a later cold-format comparator.

One broker role is selected only if multi-replica service fanout cannot remain simpler.
Zenoh requires a concrete edge topology before comparison.
tracing and OpenTelemetry become essential with runtime implementation.

EvidencePrompt: framework-and-tool evaluation.

operations

Failure, security, and observability

Failures are typed state transitions with visible watermarks, bounded consequences, and an owned recovery command.

Question: How does an operator prove the system is degraded rather than silently wrong?
Binding decision: Secrets never reach the browser; tenant, entitlement, query budget, generation, and provenance cross every boundary.
Owner input: Alert thresholds and error budgets remain proposals until production profiles are measured.

How does an operator prove the system is degraded rather than silently wrong?symptom → boundary → recovery

01Providerquota / outagepause bounded work
02QuestDBcommit laghold durable watermark
03Redisloss / trimquery authority or resync
04Ownerlease expiryfence and reacquire
05Viewerqueue pressureshed, resync, or close

Planned target gate · Exercise failure domainsmise x -- task fanout:chaos

Acceptance target: Partitions and slow consumers produce bounded, typed recovery.

Failure is observable state

The system emits traces, structured logs, and metrics for provider cost, ingest and commit watermarks, coverage gaps, query plans, cache staleness, owner epochs, broker lag, pending and reclaimed messages, viewer queue depth, reconnects, and resync outcomes.

Cardinality budgets prevent instrument IDs from exploding every metric.
Watermark lag locates the failing stage in one trace.
Owner death, Redis loss, QuestDB lag, and subscriber stall have separate recovery semantics.

EvidencePrompt: observability and backpressure analysis requirements.

The gateway owns trust

The browser receives only an authorized query and stream contract. The gateway checks tenant, entitlement, query range, rate, and concurrency before work begins. Credentials for providers, stores, brokers, and workflows remain server-side and logs redact them.

Per-tenant subjects and authorization prevent cross-tenant fanout.
Malformed and oversized requests consume bounded parser and query budgets.
Retention and data-rights decisions gate persistence and deletion.

Evidencedocs/SECURITY.md and target architecture contract.

operations

The engineering loop

Fast local feedback and deep scheduled evidence are separate tool profiles, not one bloated default installation.

Question: Which proof belongs at edit, commit, CI, security, and performance time?
Binding decision: Stable rustfmt, strict Clippy, rust-analyzer, Nextest, doctests, mise, Task, and prek form the baseline.
Owner input: Product-specific property, model, fuzz, chaos, and benchmark owners activate with the runtime slice they prove.

Which proof belongs at edit, commit, CI, security, and performance time?symptom → boundary → recovery

01Providerquota / outagepause bounded work
02QuestDBcommit laghold durable watermark
03Redisloss / trimquery authority or resync
04Ownerlease expiryfence and reacquire
05Viewerqueue pressureshed, resync, or close

Run the local blocking gatemise x -- task verify

Pinned tools, formatting, architecture, prompt, lint, tests, state, and diff integrity pass.

Essentials stay fast

The base profile pins Rust 1.97, rustfmt, Clippy, rust-analyzer, rust-src, Task, prek, jq, and Nextest. CI adds coverage and sccache. Security adds cargo-deny and cargo-audit. Developer, debug, and performance profiles activate only for their owning investigation.

Bacon watches the edit loop; cargo-expand inspects macros.
Hyperfine measures process paths; Samply and flamegraph explain profiles.
cargo-testdox communicates behavior; cargo-mutants challenges meaningful suites.

Evidencedocs/dev/testing.md and mise profile files.

Specialists need activation evidence

Proptest, fuzzing, Loom or Shuttle, Turmoil or Madsim, testcontainers, wiremock, Criterion or Divan, and iai-callgrind attach to a named risk. syn, autocfg, valuable, nom, Steel, and Lurk remain explicitly rejected from the baseline until their narrow triggers exist.

A tool names its problem, owner, command, CI placement, cost, and removal condition.
Coverage is diagnostic rather than a vanity threshold.
Benchmark changes require a representative release workload.

Evidencedocs/dev/invariants.md and docs/dev/rust-rule-map.md.

migration

Migration without a cliff

The candidate path earns traffic through shadow evidence, narrow cohorts, and exercised rollback—not a flag-day rewrite.

Question: How does the old system remain a safe exit while the new one proves itself?
Binding decision: Reads, writes, cache, replay, ownership, and frontend cutover remain independently reversible.
Owner input: Production rollout authority, cohort size, rollback window, and deletion date require explicit owner approval.

0Freezeobserve
1Skeletonbuild + gate
2QuestDBbuild + gate
3Redisbuild + gate
4Backfillbuild + gate
5Fanoutbuild + gate
6Frontendbuild + gate
7Cold tierbuild + gate
8Cutoverprove + release

rollback remains live through every phase

Planned target gate · Rehearse the exitmise x -- task migration:rollback-drill

Acceptance target: Every feature flag restores the last proven state within accepted RTO/RPO.

Compare before serving

The candidate first dual-runs canonical Nautilus values without changing client reads, then shadow-writes QuestDB and compares coverage, counts, hashes, and watermarks. Representative history queries are diffed before internal users or symbols enter a canary cohort.

Generation and resync parity precede frontend cutover.
Cache, replay, and multi-replica ownership activate independently.
Expected and 10x profiles gate each expansion.

EvidenceExecPlan Phase 8.

Deletion is a separate decision

Every flag is rolled back in rehearsal before advancing. Old coordination paths, flags, tests, and docs are deleted only after the rollback window and retention, entitlement, security, and data-rights approvals.

A failed gate stops cohort expansion immediately.
Shadow data and comparison evidence remain available for diagnosis.
Deletion never shares a commit with the first production cutover.

EvidenceExecPlan Phase 8 rollback and stop conditions.

migration

Execution map

Nine phases integrate the Nautilus kernel first, then add durable projections, browser delivery, proof, rollback, and cutover.

Question: What is the next safe unit of work?
Binding decision: Only one implementation feature is active; bounded research and review may run concurrently with exclusive outputs.
Owner input: The next active feature is created in the Tradingview target repo only after Phase 0 freezes evidence.

0Freezeobserve
1Skeletonbuild + gate
2QuestDBbuild + gate
3Redisbuild + gate
4Backfillbuild + gate
5Fanoutbuild + gate
6Frontendbuild + gate
7Cold tierbuild + gate
8Cutoverprove + release

rollback remains live through every phase

Open the orchestration sourcesed -n '1,220p' docs/generated/databento-data-layer-execplan.md

The complete phase ownership and gate contract is available.

The phase sequence

Phase 0 freezes current behavior and exact Nautilus revision/features. Phase 1 composes the Nautilus runtime, browser/service contracts, projection ports, and CLI. Phase 2 proves the bounded QuestDB projection and ReplayCatalog. Phase 3 adds rebuildable Redis projections. Phase 4 proves lazy historical fill. Phase 5 proves multi-user fanout. Phase 6 converges the frontend. Phase 7 activates a cold tier only on evidence. Phase 8 shadows, canaries, cuts over, and eventually deletes.

Every phase names exclusive paths and prerequisites.
Every phase has exact mise/Task proof commands and retained evidence.
Every phase defines rollback, stop conditions, and done.

Evidencedocs/generated/databento-data-layer-execplan.md.

The first move is observation

The design does not authorize production backend mutation. Begin in an isolated Tradingview worktree, freeze the current contract, pin the Nautilus source and feature closure, characterize DBN-to-Nautilus semantics, and write the preserve/change/delete matrix before introducing the runtime composition.

Runtime code and tests outrank documentation for current behavior.
Unknowns that change correctness, security, cost, or irreversible state stop the phase.
After two failed approaches at one seam, reduce to a minimal comparison spike.

EvidenceExecPlan Phase 0 and stop/escalation rules.