Architecture

How zecd is put together: a single-writer actor per wallet database, a read path that bypasses it, a sync engine sliced into batches so the actor stays responsive, the background loops (enhancement, mempool, rebroadcast) that hang off the sync loop, and one shared upstream connection per endpoint that every actor talks through. The Zebra backend and statelessness pages cover the upstream interface and the persistence invariant separately.

Component diagram

 RPC client (python-bitcoinrpc, curl, ...)
      |
      v  HTTP Basic / cookie auth
+---------------------------+        +------------------+
| axum RPC server           |        | health server    |
| auth gate -> work-queue   |        | /healthz /readyz |
| semaphore -> dispatch     |        | /status          |
+------------+--------------+        +---------^--------+
             |                                 | SyncStatus (watch channel)
     per-wallet WalletHandle                   |
      |                   |                    |
      | reads             | writes             |
      v                   v                    |
+-----------------+   mpsc command channel     |
| short-lived     |   (oneshot reply each)     |
| SQLite conns    |          |                 |
| (WAL snapshots) |          v                 |
+--------+--------+   +------+----------------------+
         |            | WalletActor (single writer) |
         |            |  owns WalletDb (data.sqlite)|
         +----------->|  sync loop / enhance /      |
           same DB    |  mempool / rebroadcast /    |
           file       |  sends (prove + broadcast)  |
                      +------+----------------------+
                             |  HubSource (a ChainSource)
                             v
                      +------+----------------------+
                      | ChainHub (one per endpoint) |
                      |  shared by every actor      |
                      +------+----------------------+
                             |  AnySource -> ZebraSource / light source
                             v
                       zebrad JSON-RPC (zebra://host:port) or lightwalletd

A fleet shard is one more actor of the same kind: one database holding many watch-only accounts, scanned in one pass.

The single-writer actor

zcash_client_sqlite::WalletDb is Send but not Sync, and wallet writes (note selection, scan application, stores) must not interleave. zecd therefore gives each configured wallet one actor task (src/wallet/actor.rs) that owns the WalletDb and is the only writer. It is the analog of Bitcoin Core's cs_wallet mutex (src/wallet/wallet.h in the Core tree): every state-changing operation serializes through one queue, so two concurrent sendtoaddress calls cannot select the same notes and double-spend. The actor also runs the background sync loop, so scans and sends contend for the same writer by construction.

RPC handlers talk to the actor through a clonable WalletHandle over a bounded tokio mpsc channel (capacity 64); each WalletCommand carries a oneshot reply sender the handler awaits. The command set is small: GetNewAddress, GetAddressForAccount, Send, GetRawTx, Broadcast, Unlock, Lock. Everything else is a read.

Read-only RPCs (getbalance, listtransactions, listunspent, getwalletinfo, ...) never enter the queue. The wallet DB runs in WAL mode, so src/wallet/read.rs serves them from short-lived read connections with consistent snapshots. Reads keep working during a long scan or proof, and a wedged writer cannot block balance queries. Two more things bypass the queue deliberately: sync state is published on a watch channel (SyncStatus), read lock-free by the blockchain RPCs and the health server; and walletlock zeroizes the shared seed directly from the handle, so the seed does not linger behind a queued long send.

Command handling and mempool ingestion are panic-isolated (catch_unwind): a poison transaction or a librustzcash edge case fails that one command instead of killing the actor, which would silently stop all writes while reads kept answering.

The actor's main loop, in order per pass: drain any finished pipelined sends, drain all queued commands (writers are never starved by sync), then run one sync batch. When the batch reports no more work (caught up), it runs the rebroadcast pass, one enhancement batch, and (re)opens the mempool subscription. Idle, it sits in a select! over shutdown, commands, send completions, the relock deadline, the poll tick ([sync] interval_secs, default 20), and the mempool stream.

The sync engine

src/sync/engine.rs is the compact-block scan loop, ported from zcash-devtool and refactored into a one-batch-per-call driver: sync_one_batch downloads and scans up to [sync] batch_size compact blocks (default 10,000), then returns to the actor loop so queued commands run between batches. Since 0.8.0 the batch is held in memory (sync/memcache.rs) rather than written one file per block, and for shielded-only wallets the next range downloads while the current one scans. A monolithic run-until-caught-up loop (librustzcash ships one behind its sync feature) would hold the writer for the whole initial sync; it also leaves the RequestedRewindInvalid reorg case unhandled, which is why zecd keeps its own driver.

Reorg detection is librustzcash's (a continuity error from scan_cached_blocks); recovery is caller-side by upstream design. zecd's perform_rewind truncates below the conflict and retries at a shallower height when the requested rewind is invalid, so young wallets survive reorgs near their birthday. Since 0.8.0 the rewind margin starts at 10 blocks and doubles for each consecutive reorg, up to one batch, so a deep rollback is walked back in a few rounds rather than dozens; it returns to 10 after the first clean batch. Compact blocks themselves are derived from full zebrad blocks; see the Zebra backend. For transparent-enabled wallets the same pass matches each scanned block's transparent outputs against the wallet's exposed-address set; see transparent support.

Transaction enhancement

Compact blocks carry no memos and no full transaction data, so the block scan records a received note with a NULL memo. enhance_step backfills this by servicing librustzcash's transaction_data_requests: for each request it fetches the full transaction from zebrad and runs decrypt_and_store_transaction, recovering received memos and (via the sender's OVK) the wallet's own outgoing memos. Without it, any transaction the wallet only ever saw as a compact block (every receive during initial sync or a restore) would never show its memo.

On a from-birthday restore the backlog is one upstream fetch per transaction: potentially tens of thousands of requests after the block scan already reached the tip. So enhance_step is bounded like the scan: one pass per call, [sync] enhance_concurrency requests (default 16), with commands serviced and the shrinking backlog republished between passes. Since 0.8.0 a pass fetches its requests concurrently, reads the request table once, and commits once, so a pass records everything it fetched or none of it. [sync] fetch_memos = false skips the memo fetches entirely (see configuration). The count rides on SyncStatus.pending_enhancements and is an observable readiness signal: while it is non-zero the wallet reports getwalletinfo.scanning: true, /readyz returns 503 with reason: "enhancing" in synced mode, and /status shows the number. "Scanned to tip" is not "ready to serve full history"; see operations for monitoring it.

The count is of distinct outstanding requests. The upstream query that generates the recurring transparent spend-search requests joins its per-UTXO queue to received outputs on transaction id alone, so a transaction with k unspent wallet outputs yields k identical requests per queue row; deduplicating is what keeps the published count meaningful and stops the drain servicing the same request repeatedly within one batch. Because those requests re-emit whenever the tip passes an output's observed height, the backlog also rises in steady state on a wallet holding many transparent UTXOs, not only during a restore, which is what readiness = "scanned" exists for.

Mempool poller (0-conf)

Once caught up, the actor subscribes to the upstream mempool stream. Since 0.8.0 there is one stream per upstream, owned by the chain hub and fanned out to every actor, rather than one per wallet: a 2-second getrawmempool poller that closes itself when getbestblockhash changes, which doubles as the "new block, sync now" signal. Every mempool transaction is processed twice over: it is trial-decrypted against the wallet's keys (decrypt_and_store_transaction is a no-op for unrelated transactions), and its transparent outputs are matched against the exposed-address set. Incoming payments of either kind therefore appear at 0 conf in getunconfirmedbalance, listtransactions, and listunspent minconf=0, as in bitcoind. The subscription is best-effort: a stream error just drops it until the next caught-up pass. The actor also stamps a transient in-memory first-seen time for unmined transactions here (surfaced as time/timereceived); it is never persisted, per the statelessness invariant.

Rebroadcast loop

On caught-up passes, at most once per [sync] rebroadcast_secs (default 60), the actor re-submits wallet transactions that are still unmined and unexpired. Only transactions that spend this wallet's own notes or UTXOs qualify: nobody else can spend them, so they were necessarily authored here, and foreign unmined transactions the mempool stream stored are the sender's to retransmit. A node that already holds the transaction rejects the duplicate, which is logged at debug and harmless. This is what makes sendtoaddress safe to return a txid even when the initial relay fails: the inputs are locked in the DB until expiry and the loop keeps retrying.

The spend path

All sends (sendtoaddress, sendmany, z_sendmany) funnel through the actor's do_send. Three details matter to operators:

sync_to_tip_for_send. librustzcash sets a transaction's target height (and thus its expiry) and its spend anchor from the wallet DB's recorded chain tip. If the sync loop has starved under load, that tip can lag Zebra's real tip past the expiry delta, and the send is rejected upstream as already expired (-25, intermittently). Bumping only the tip pointer is worse: the anchor then falls in an unscanned range and get_wallet_summary zeroes the entire shielded balance, turning the failure into -6 ("0 spendable"). So before building, do_send refreshes the tip and drives sync_step until the tip captured at entry is scanned. Normally a no-op; best-effort (an unreachable upstream falls back to the last-scanned height).

Cached Orchard proving key. With [spend] cache_proving_key (on by default), sends run through the PCZT roles with an orchard::circuit::ProvingKey shared by Arc across all actors. Since 0.8.0 the Zakura Common crates also cache the keys process-wide on the fused path (flag off), so neither path rebuilds a key per transaction; the flag now chooses only whether the keys are warmed at startup and whether pipeline_proving can engage. Key generation takes well under a second, and is skipped at startup when no loaded wallet can spend. Proving runs under tokio::task::block_in_place, so it does not stall the async runtime, but it does hold the actor.

Since 0.6.0 the key is built off the startup critical path. daemon::run kicks off the build and carries straight on to spawn wallet actors and bind the listeners; the first send awaits the handle, which on any real deployment resolves immediately because the build finished long before. The handle is a tokio::sync::OnceCell, so concurrent senders wait on the one in-flight build rather than starting a second, and a send arriving before the background task ran simply drives the build itself. A keygen panic now fails that send rather than the whole daemon. The Orchard and Ironwood keygens are independent, so they run on separate threads.

[spend] pipeline_proving (default off). By default the whole send (select, build, prove, sign, store, broadcast) runs on the actor, so a long proof freezes background sync for its duration. With pipelining on, the send splits: phase A (note selection + PCZT build, a milliseconds-scale DB read) stays on the actor, phase B (prove + sign) runs on a blocking thread, and phase C (extract + store + broadcast) returns to the actor. Sends still serialize: only one PCZT is ever uncommitted, so there is no double-spend surface; a send arriving mid-proof queues (up to 64, then -4 back-pressure) and starts when the in-flight one commits. It improves liveness, not multi-send throughput. Engages only on the cached-Orchard PCZT path. Every send logs a phase profile (select+build / prove+sign / store / broadcast milliseconds plus input and action counts).

Datadir lock

The single-writer invariant holds within one process; a second zecd on the same datadir would still corrupt the wallet DB. src/lock.rs takes an exclusive advisory lock on <datadir>/.lock (via fmutex, as zcashd does). zecd (the daemon) and zecd init take it and hold it for their lifetime; a second writer fails to start with "Cannot lock data directory ... Another zecd is already running". zecd rpcauth (no datadir access) and zecd export-ufvk (read-only) deliberately do not take it, so the UFVK stays exportable while the daemon runs.

Module map