Dedicated snapshot-builder service (mainnet)

A stand-alone node whose only job is to build a snapshot of the chain every 6 hours and upload it to Cloudflare R2. It serves no RPC, signs no blocks, and is frozen while it is copied, so the copy is consistent and nothing user-facing depends on it.

History: the first setup uploaded from archive-snapshot-publisher, the node that also serves archive.sanect.com. Two problems, both fixed here:

1. The cron never ran. jq '.catching_up // true' turns false into true (jq's // treats false as empty), so it waited forever (waiting for catching_up=false ... waited 87000s).
2. Copying a live LevelDB (tar: ...ldb: File removed before we read it) can produce a snapshot with missing files. Freezing the process (SIGSTOP) during the copy is only acceptable on a node that serves nothing, so that is now a separate service.

Architecture

 primary validator --p2p--> archive-snapshot-publisher (PRUNING=nothing, serves archive.sanect.com)
                                   |  state-sync over the private network (no load on the validator)
                                   v
                          snapshot-builder (this doc: pruned node, no domain)
                                   |  every 6 h: SIGSTOP node -> tar | lz4 | rclone (64-128 MiB parts) -> SIGCONT
                                   v
                       Cloudflare R2  sanect-snapshots-mainnet  ->  https://snapshots.sanect.com/LATEST.json

The snapshot is of a pruned node (PRUNING=custom: last 20,000 state versions, last 100,000 blocks). That is what a validator or RPC joiner needs, and it is a fraction of the size of the archive data dir. If you tune it, know where the size goes: in one measured snapshot (871 MB raw) application.db was 557 MB, blockstore.db 117 MB, tx_index.db 106 MB and state.db 86 MB, so the state history setting matters most. Archive-node operators still sync from peers.

Railway setup

1. New service from the repo (GitHub repo sanect/sanect-node, branch main).
2. Root directory empty; Dockerfile docker/Dockerfile (same image as the validators).
3. Volume mounted at /data, 50 GB to start. A fresh volume, never shared (a copied validator volume would copy its consensus key).
4. No public domain. Do not generate one.
5. Plan: 4 vCPU / 8 GB RAM is plenty (keep MEMLIMIT_GIB below ~70 % of RAM).

Env vars

# Chain identity
CHAIN_ID=sanect_7628-1
EVM_CHAIN_ID=7628
HOMEDIR=/data/.sanectd
MONIKER=snapshot-builder

# Join through the ARCHIVE node over Railway's private network, not the validator
JOIN_NETWORK=true
SEED_NODE_URL=http://archive-snapshot-publisher.railway.internal:8080
SEED_NODE_ID=2fb2bcba86403ef8a14f7cfc236a2462f0a0faa0
SEED_PEER_HOST=archive-snapshot-publisher.railway.internal:26656
# Re-check the id any time:
#   curl -s https://archive.sanect.com/rpc/status | jq -r .result.node_info.id

# Pruned snapshot (small, what joiners need)
PRUNING=custom
# State history to keep. This is what makes the snapshot big: measured 2026-10-04,
# application.db was 557 of 871 MB (100,000 versions kept). Joiners only need the
# latest state, so keep far less here.
PRUNING_KEEP_RECENT=20000
PRUNING_INTERVAL=100
# Block store: drop blocks older than this (default: keep every block). 100000 equals
# the consensus evidence window, so anything started from this snapshot stays valid.
MIN_RETAIN_BLOCKS=100000

# Keep memory low: the entrypoint default (10,000,000) is a load-test value
IAVL_CACHE_SIZE=781250
MEMLIMIT_GIB=6

# Makes the node unable to validate (entrypoint deletes priv_validator_key.json)
SNAPSHOT_PUBLISHER=true

# Upload schedule + speed
SNAPSHOT_UPLOAD_INTERVAL_HOURS=6
SNAPSHOT_KEEP=3
SNAPSHOT_FREEZE=true            # SIGSTOP the node while tar reads the data dir
SNAPSHOT_LOW_PRIORITY=false     # dedicated node: no nice/ionice
SNAPSHOT_CHUNK_SIZE=128M        # default 64M. rclone's 5 MiB default is slow and caps a file at 48.8 GiB
SNAPSHOT_UPLOAD_CONCURRENCY=8   # parallel parts; extra RAM is about chunk x concurrency (1 GiB here)

# Cloudflare R2 (only on THIS service)
R2_ACCOUNT_ID=<32-char Cloudflare account id>
R2_ACCESS_KEY_ID=<R2 token access key>
R2_SECRET_ACCESS_KEY=<R2 token secret>
R2_BUCKET=sanect-snapshots-mainnet
R2_PUBLIC_URL=https://snapshots.sanect.com

Do not set EXPLORER_HEARTBEAT_URL, PUBLIC_RPC_URL, or SNAPSHOT_UPLOAD_ALLOW_ON_VALIDATOR, and never mount a priv_validator_key.json.

Remove the upload env from the archive service

So two services never write the same LATEST.json, on archive-snapshot-publisher delete: R2_ACCOUNT_ID, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET, R2_PUBLIC_URL, SNAPSHOT_UPLOAD_INTERVAL_HOURS, SNAPSHOT_KEEP, and any SNAPSHOT_FREEZE / SNAPSHOT_CHUNK_SIZE / SNAPSHOT_UPLOAD_CONCURRENCY you set. Keep PRUNING=nothing and SNAPSHOT_PUBLISHER=true there. Never set SNAPSHOT_FREEZE=true on a node that serves RPC.

R2 public access (why snapshots.sanect.com returned 403)

Bucket sanect-snapshots-mainnet -> Settings -> Custom domains -> connect snapshots.sanect.com (status Active), and allow public access for it. Then check Cloudflare -> Security -> Events for any WAF rule blocking it. A missing key should return 404, not 403.

First run, and how to verify

The first upload starts about 30 s after the node catches up (the initial state sync takes minutes). To force one immediately: Railway -> this service -> Console:

bash /app/scripts/snapshot-upload.sh --force

Log lines to look for: [snapshot-cron] node is caught up, freezing sanectd, resumed sanectd, manifest published.

curl -s https://snapshots.sanect.com/LATEST.json | jq
# height near tip, created_at within the last 6 h, chain_id sanect_7628-1

Test a restore before telling anyone to use it. On a throwaway node set SNAPSHOT_MANIFEST_URL=https://snapshots.sanect.com/LATEST.json (plus the normal JOIN_NETWORK vars from the public run-a-node guide) and confirm it reaches the tip. If the download or restore fails the entrypoint falls back to state sync.

Cost

Railway about $10-20/month for the small node and volume; R2 about $1-5/month for three pruned snapshots (R2 has no egress fees).

---

Archive snapshots: a second builder, kept apart (optional)

Archive snapshots are for operators who want a node with all history. They are about 80 GB compressed (versus hundreds of MB for the pruned snapshot), so they get their own service, their own folder in the bucket and their own manifest. They must never share LATEST.json or the "keep the newest N" pruning with the pruned snapshots.

	pruned (snapshot-builder)	archive (snapshot-builder-archive)
Objects	sanect-<height>.tar.lz4	archive/sanect-<height>.tar.lz4
Manifest	https://snapshots.sanect.com/LATEST.json	https://snapshots.sanect.com/archive/LATEST.json
Size	hundreds of MB	about 80 GB
Pruning	custom (last 100,000 blocks)	nothing
Schedule	every 6 h	every 24 h
Node history	recent only (state sync)	from block 1

Why it cannot just state-sync

A node that joins now would state-sync to a recent height and have no history, even with PRUNING=nothing. The archive builder therefore needs either DISABLE_STATE_SYNC=true (syncs every block from genesis; slow) or to be seeded from an existing archive snapshot with SNAPSHOT_URL (fast; the node then only syncs the blocks since the snapshot).

Railway setup

New service from the repo (main, docker/Dockerfile), build arg TARGET_EVM_CHAIN_ID=7628, a fresh volume at /data (300 GB or more), no domain, about 16 GB RAM. Variables:

CHAIN_ID=sanect_7628-1
EVM_CHAIN_ID=7628
HOMEDIR=/data/.sanectd
MONIKER=snapshot-builder-archive

# join through the archive node over the private network
JOIN_NETWORK=true
SEED_NODE_URL=http://archive-snapshot-publisher.railway.internal:8080
SEED_NODE_ID=2fb2bcba86403ef8a14f7cfc236a2462f0a0faa0
SEED_PEER_HOST=archive-snapshot-publisher.railway.internal:26656

PRUNING=nothing
DISABLE_STATE_SYNC=true
# First boot only (then remove it): restore an existing archive snapshot instead of
# syncing 16M+ blocks. Use a snapshot that is known to restore.
#SNAPSHOT_URL=https://snapshots.sanect.com/sanect-<height>.tar.lz4
# Archive nodes must NOT fall back to replaying from block 1 if the download fails:
# fail the boot instead, so the platform retries the download from scratch.
SNAPSHOT_REQUIRED=true
# Download mode: auto picks "file" (resumable HTTP Range download to /data/snapshot-download.part,
# then extract) when the volume has about 2.6x the snapshot size free, else "stream". File mode is
# required for big snapshots: the connection to the snapshot host is cut after ~9 min / ~22 GB
# (curl 18 / 92), so one uninterrupted 80 GB stream never finishes, and the .part resumes across
# restarts. Keep the archive builder volume at 300 GB or more.
#SNAPSHOT_DOWNLOAD_MODE=auto
#SNAPSHOT_DOWNLOAD_ATTEMPTS=60
# Optional: allow slow links more time (default 21600 s = 6 h; the old fixed limit of
# 30 min could not fit an 80 GB download at ~41 MB/s).
#SNAPSHOT_DOWNLOAD_MAX_TIME=21600

SNAPSHOT_PUBLISHER=true
SNAPSHOT_FREEZE=true
SNAPSHOT_LOW_PRIORITY=false
SNAPSHOT_CHUNK_SIZE=128M
SNAPSHOT_UPLOAD_CONCURRENCY=8
SNAPSHOT_UPLOAD_INTERVAL_HOURS=24
SNAPSHOT_KEEP=2
SNAPSHOT_PREFIX=archive/

MEMLIMIT_GIB=12
IAVL_CACHE_SIZE=781250

# same R2 credentials and bucket as the pruned builder
R2_ACCOUNT_ID=<32-char Cloudflare account id>
R2_ACCESS_KEY_ID=<R2 token access key>
R2_SECRET_ACCESS_KEY=<R2 token secret>
R2_BUCKET=sanect-snapshots-mainnet
R2_PUBLIC_URL=https://snapshots.sanect.com

While the node is frozen for the copy (the whole upload, hours) it falls behind; afterwards it catches up in minutes. That is fine because it serves no RPC.

Order of operations when splitting the services

1. Stop the archive RPC node from uploading: on archive-snapshot-publisher delete R2_ACCOUNT_ID, R2_ACCESS_KEY_ID, R2_SECRET_ACCESS_KEY, R2_BUCKET, R2_PUBLIC_URL, SNAPSHOT_UPLOAD_INTERVAL_HOURS, SNAPSHOT_KEEP. Redeploy.
2. Raise SNAPSHOT_KEEP on snapshot-builder to 6 for now, so its pruning does not delete the old 80 GB seed file before the archive builder has downloaded it.
3. Repoint the pruned manifest: in snapshot-builder's Console run bash /app/scripts/snapshot-upload.sh --force, then check that https://snapshots.sanect.com/LATEST.json shows a small size_bytes.
4. Create snapshot-builder-archive as above, seeded with the 80 GB file.
5. When it has uploaded its first archive/ snapshot, delete the old 80 GB root objects and put SNAPSHOT_KEEP back to 3 on the pruned builder.

Operators who want an archive node then use SNAPSHOT_MANIFEST_URL=https://snapshots.sanect.com/archive/LATEST.json with PRUNING=nothing and DISABLE_STATE_SYNC=true.