diff --git a/README.md b/README.md index cd10ddc4..19d1332b 100644 --- a/README.md +++ b/README.md @@ -13,13 +13,13 @@ nodecore is **API/protocol-agnostic**: it is not tied to a single RPC shape. It ### Supported chains, methods & interfaces - **Interfaces** — `json-rpc` (over HTTP), `websocket`, and `rest` upstream connectors. The set available for a given chain is declared by that chain's [method spec](docs/nodecore/11-method-specs.md). -- **Chains** — every chain defined in [`chains.yaml`](https://github.com/drpcorg/public/blob/main/chains.yaml). Current chain families are EVM (Ethereum, Polygon, Optimism, Arbitrum, Base, BSC, and many others), Solana, Algorand, Aztec, Aptos, and the Ethereum/Gnosis Beacon Chain (consensus layer, REST-only). +- **Chains** — every chain defined in [`chains.yaml`](https://github.com/drpcorg/public/blob/main/chains.yaml). Current chain families are EVM (Ethereum, Polygon, Optimism, Arbitrum, Base, BSC, and many others), Solana, Algorand, Aztec, Aptos, NEAR, and the Ethereum/Gnosis Beacon Chain (consensus layer, REST-only). - **Methods** — per-chain RPC behavior is data-driven via [method specs](docs/nodecore/11-method-specs.md), covering standard EVM/Solana methods, subscriptions, and EVM filter methods. Methods unsupported by an upstream are automatically banned for it to avoid wasted requests. ## Key features - **Intelligent routing** — dynamically selects the most suitable upstream based on real-time performance metrics (latency, error rate, availability) for optimal speed, reliability, and fault-tolerance. See [Upstream config](docs/nodecore/05-upstream-config.md). -- **API/protocol-agnostic** — JSON-RPC, WebSocket, and REST interfaces across EVM, Solana, Algorand, Aztec, Aptos, and the Ethereum/Gnosis Beacon Chain, all driven by data-defined [method specs](docs/nodecore/11-method-specs.md). EVM filter methods (`eth_newFilter`, `eth_getFilterLogs`, etc.) are routed only to the upstream where the filter was created. +- **API/protocol-agnostic** — JSON-RPC, WebSocket, and REST interfaces across EVM, Solana, Algorand, Aztec, Aptos, NEAR, and the Ethereum/Gnosis Beacon Chain, all driven by data-defined [method specs](docs/nodecore/11-method-specs.md). EVM filter methods (`eth_newFilter`, `eth_getFilterLogs`, etc.) are routed only to the upstream where the filter was created. - **Subscriptions** — WebSocket subscriptions are aggregated so many identical client subscriptions share one upstream stream, with optional local synthesis of EVM topics (`newHeads`, `logs`, pending transactions). See [Subscriptions](docs/nodecore/13-subscriptions.md). - **Caching** — minimizes redundant traffic by caching frequent requests across in-memory/Redis/Postgres backends with configurable policies. See [Cache](docs/nodecore/04-cache.md). - **Failsafe mechanisms** — request hedging (duplicate slow requests to multiple upstreams) and configurable automatic retries. See [Upstream config](docs/nodecore/05-upstream-config.md). diff --git a/docs/nodecore/05-upstream-config.md b/docs/nodecore/05-upstream-config.md index 19357aeb..29e952b9 100644 --- a/docs/nodecore/05-upstream-config.md +++ b/docs/nodecore/05-upstream-config.md @@ -337,8 +337,8 @@ The `chain-defaults` section defines per-chain baseline settings. `.optio * `disable-health-validation` - Disables only the health validators (per chain family). **_Default_**: `false` * `disable-lower-bounds-detection` - Disables the earliest-available-block detector. Mode-dependent default: `true` in `default` mode, `false` in `strict` mode * `disable-labels-detection` - Disables the EVM label detectors (client/version, archive, gas, flashblock, etc.). Mode-dependent default: `true` in `default` mode, `false` in `strict` mode - * `validate-syncing` - For EVM chains, calls `eth_syncing` periodically and marks the upstream unavailable when it is syncing. For beacon-chain upstreams it probes `GET /eth/v1/node/syncing` instead (marking the upstream `Syncing`, or `Unavailable` when its execution layer is offline). Mode-dependent default: `false` in `default` mode, `true` in `strict` mode - * `validate-peers` - For EVM chains, calls `net_peerCount` periodically and pairs with `min-peers`. For beacon-chain upstreams it probes `GET /eth/v1/node/peer_count`. Mode-dependent default: `false` in `default` mode, `true` in `strict` mode + * `validate-syncing` - For EVM chains, calls `eth_syncing` periodically and marks the upstream unavailable when it is syncing. For beacon-chain upstreams it probes `GET /eth/v1/node/syncing` instead (marking the upstream `Syncing`, or `Unavailable` when its execution layer is offline). For NEAR upstreams the `status` probe's `sync_info.syncing` and a stale-head guard on `latest_block_time` drive the same signal (always on as part of health validation). Mode-dependent default: `false` in `default` mode, `true` in `strict` mode + * `validate-peers` - For EVM chains, calls `net_peerCount` periodically and pairs with `min-peers`. For beacon-chain upstreams it probes `GET /eth/v1/node/peer_count`. NEAR upstreams probe `network_info` `num_active_peers`. Mode-dependent default: `false` in `default` mode, `true` in `strict` mode * `min-peers` - Minimum acceptable peer count when `validate-peers` is on. **_Default_**: `1` * `validate-call-limit` - For EVM chains, periodically probes the upstream's `eth_call` return-data limit and marks the upstream unhealthy when its observed limit is below `call-limit-size`. Mode-dependent default: `false` in `default` mode, `true` in `strict` mode * `call-limit-size` - Threshold (in bytes) of the smallest acceptable `eth_call` return-data limit. **_Default_**: `1000000` (1 MB) @@ -674,6 +674,7 @@ Validators and label detectors run periodically (every `validation-interval`) ag | Chain id / `net_version` | EVM | `disable-chain-validation`, `disable-settings-validation`, `disable-validation` | Confirms the upstream is actually serving the configured chain. Fails at startup remove the upstream from the pool; runtime drift triggers re-removal | | Aztec chain validator | Aztec | `disable-chain-validation`, `disable-settings-validation`, `disable-validation` | Equivalent of chain-id check, using the Aztec node's chain-id endpoint | | Aptos chain validator | Aptos | `disable-chain-validation`, `disable-settings-validation`, `disable-validation` | Equivalent of chain-id check, using the `chain_id` from the Aptos ledger-info endpoint (`GET /v1`) | +| NEAR chain validator | NEAR | `disable-chain-validation`, `disable-settings-validation`, `disable-validation` | Equivalent of chain-id check, comparing the `status` probe's `chain_id` against the configured chain | | `eth_syncing` validator | EVM | `validate-syncing` (set to `false`) or `disable-settings-validation` | Marks the upstream as syncing/unavailable when the node reports it is not fully synced | | `net_peerCount` validator | EVM | `validate-peers` / `min-peers` or `disable-settings-validation` | Marks the upstream as unhealthy when peer count drops below `min-peers` | | `eth_call` return-data limit | EVM | `validate-call-limit` or `disable-settings-validation` | Probes the upstream's maximum `eth_call` return-data size and marks it unhealthy if it is below `call-limit-size` | @@ -685,10 +686,13 @@ Validators and label detectors run periodically (every `validation-interval`) ag | Health validator (Beacon) | Beacon Chain | `disable-health-validation` | Calls `GET /eth/v1/node/health` and marks the upstream available only on a 2xx response | | Syncing validator (Beacon) | Beacon Chain | `validate-syncing` (set to `false`) | Probes `GET /eth/v1/node/syncing`; marks the upstream `Syncing` when `is_syncing` is true and `Unavailable` when `el_offline` is true | | Peers validator (Beacon) | Beacon Chain | `validate-peers` / `min-peers` | Probes `GET /eth/v1/node/peer_count` and marks the upstream immature while connected peers are below `min-peers` | +| Health validator (NEAR) | NEAR | `disable-health-validation` | Calls the NEAR `status` RPC; marks the upstream `Syncing` when `sync_info.syncing` is true or `latest_block_time` is stale (stale-head guard). With `validate-peers` on, also probes `network_info` and marks the upstream `Unavailable` at zero `num_active_peers` | | Lower-bound detector | Solana, Algorand, Aztec, Aptos | `disable-lower-bounds-detection` | Determines the earliest available block / slot on the upstream so that queries against pruned ranges can be routed away | | Lower-bound detector (Beacon) | Beacon Chain | `disable-lower-bounds-detection` | Binary-searches the earliest retained block, state, epoch (attestation rewards), and blob-sidecar slots so requests against pruned ranges are routed away | +| Lower-bound detector (NEAR) | NEAR | `disable-lower-bounds-detection` | Reads `sync_info.earliest_block_height` from `status` and publishes it as the state and block lower bounds (a sliding GC window on non-archival nodes) | | Label detectors (EVM) | EVM | `disable-labels-detection` | Populates upstream labels - client name & version, archive vs. full, gas limit, flashblock support, high-latency-tx capability. Labels are exposed via the [gRPC API](12-grpc-server.md) so external consumers can target upstreams with specific capabilities | | Label detectors (Aptos) | Aptos | `disable-labels-detection` | Populates client name & version labels from the ledger-info endpoint (`GET /v1`) | | Client label detector (Beacon) | Beacon Chain | `disable-labels-detection` | Reads `GET /eth/v1/node/version` and publishes the consensus-client type and version labels (Lighthouse, Prysm, Teku, Nimbus, etc.) | +| Client label detector (NEAR) | NEAR | `disable-labels-detection` | Reads `version.version` from `status` and publishes the client (`neard`) and version labels | `disable-validation` is the master switch and overrides every per-validator flag. diff --git a/docs/nodecore/11-method-specs.md b/docs/nodecore/11-method-specs.md index ffb6b0c8..76079076 100644 --- a/docs/nodecore/11-method-specs.md +++ b/docs/nodecore/11-method-specs.md @@ -171,6 +171,7 @@ nodecore embeds the specs below (see [`pkg/methods/specs/`](../../pkg/methods/sp | `klaytn` | `klaytn-json-rpc`, `klaytn-websocket` | | `hyperliquid` | `hyperliquid-eth`, `hyperliquid-rest-additional` | | `tron` | `tron-json-rpc`, `tron-rest`, `tron-rest-solidity` | +| `near` | `near-json-rpc` | ### Plain specs @@ -179,7 +180,7 @@ Grouped by the transports they declare: | `api-connectors` | Specs | | --- | --- | | `json-rpc`, `websocket` | `arbitrum`, `cronos_zkevm`, `eth-json-rpc`, `fantom`, `filecoin`, `harmony_0`, `harmony_1`, `hyperliquid-eth`, `klaytn-json-rpc`, `linea`, `mantle`, `optimism`, `polygon`, `polygon_zkevm`, `rootstock`, `scroll`, `sei`, `solana-json-rpc`, `viction`, `zk` | -| `json-rpc` | `algorand`, `aztec`, `tron-json-rpc` | +| `json-rpc` | `algorand`, `aztec`, `near-json-rpc`, `tron-json-rpc` | | `websocket` | `eth-websocket`, `klaytn-websocket`, `solana-websocket` | | `rest` | `aptos`, `eth-beacon-chain`, `tron-rest` | | `rest-additional` | `hyperliquid-rest-additional`, `tron-rest-solidity` | diff --git a/docs/superpowers/specs/2026-07-17-near-support-design.md b/docs/superpowers/specs/2026-07-17-near-support-design.md new file mode 100644 index 00000000..ae5df01d --- /dev/null +++ b/docs/superpowers/specs/2026-07-17-near-support-design.md @@ -0,0 +1,176 @@ +# NEAR family support — design + +Date: 2026-07-17 +Status: approved (pre-implementation) + +## Goal + +Add first-class support for the **NEAR** blockchain family to nodecore +(`BlockchainType = "near"`), covering near mainnet and testnet (betanet is +registered in `chains.yaml` and gets family support for free), with full gRPC +(emerald) API support, so these chains can be served by nodecore instead of +dshackle. + +## Scope + +- **In scope:** neard JSON-RPC 2.0 upstreams (plain HTTP, no auth, no WS — + nearcore has no subscriptions at all); the 14-method surface currently + served by dshackle (see method table); optimistic head tracking; chain-id + validation; syncing/peers health; client version labels; dynamic GC-window + lower bounds. +- **Out of scope (YAGNI):** stable aliases nearcore added later + (`changes`, `genesis_config`, `block_effects`, `maintenance_windows`, + `light_client_proof`) and `broadcast_tx_async`/`broadcast_tx_commit`/ + `health`/`client_config`/light-client methods — none are served by dshackle + today, so no client depends on them through us; per-method cache tag + parsers (v1 ships everything `cacheable: false`, parity with dshackle which + does not cache NEAR); a separately tracked finalized head (see Head + tracking); archival-aware routing (we run no archival NEAR nodes). + +## Approach + +NEAR is modelled as a JSON-RPC blockchain family with a poll-only head and +a data-driven method spec: a `near_specific` chain-specific package plus +`pkg/methods/specs/near.json`. It is strictly simpler than bitcoin — a single +`status` call powers health, chain validation, labels and lower bounds; there +is no esplora-style secondary connector and no method aliases to translate. + +### Verified live API shapes (from our production nodes, nearcore 2.13.1) + +`status` (truncated): `{"chain_id":"mainnet","protocol_version":84, +"latest_protocol_version":86,"version":{"version":"2.13.1","build":"unknown", +"rustc_version":"1.93.0"},"validator_account_id":null,"sync_info":{ +"latest_block_height":207365026,"latest_block_hash":"...", +"latest_block_time":"2026-07-17T...","earliest_block_height":207157933, +"syncing":false},...}` — testnet identical shape with `"chain_id":"testnet"`. + +`block {"finality":"optimistic"}` → `{"author":...,"header":{"height":..., +"hash":"","prev_hash":"","timestamp":1784299563227484539, +"timestamp_nanosec":"1784299563227484539",...},"chunks":[...]}`. The +optimistic head runs ~3 blocks ahead of `"final"`. `timestamp` is integer +nanoseconds (int64-boundary risk in some parsers; `timestamp_nanosec` is the +string twin). + +`gas_price [null]` → `{"gas_price":"100000000"}` (string). + +Errors carry a structured `name`/`cause` pair alongside the standard fields: + +```json +{"error":{"name":"HANDLER_ERROR","cause":{"name":"UNKNOWN_BLOCK","info":{}}, + "code":-32000,"message":"Server error","data":"DB Not Found Error: ..."}} +``` + +- Unknown method → `REQUEST_VALIDATION_ERROR`/`METHOD_NOT_FOUND`, `-32601`. +- Garbage-collected block (`block`/`chunk`) → `UNKNOWN_BLOCK` (`query` + returns the more explicit `GARBAGE_COLLECTED_BLOCK`). +- **Unknown transaction**: the node holds the request ~10s (server-side + poll loop, 10s default timeout) and returns HTTP **408** with + `cause.name: "TIMEOUT_ERROR"` — this is not a node-health signal and must + not be blindly retried in a tight loop. +- Syncing node → `NOT_SYNCED_YET` / `NO_SYNCED_BLOCKS` (retryable on another + upstream); user errors (`INVALID_TRANSACTION`, `PARSE_ERROR`, ...) are + deterministic. + +## Architecture + +### Factory and type plumbing + +- `upstream_factory.go`: new `case chains.Near` returning + `near_specific.NewNearChainSpecificObject(...)`. +- `chains.go`: `getMethodSpecName` gains `case Near: return "near"` + (currently resolves to `""`). `chains.Near` and the NEAR/NEAR_TESTNET/ + NEAR_BETANET constants, `chains.yaml` entries (`chain-id: mainnet|testnet| + betanet`, lags 40/20, 1s block time) and gRPC ChainRefs (1050/10064/10065) + all already exist — no registry work. + +### Head tracking (`near_specific`) + +Poll-based `RpcHead` (no subscriptions), dshackle parity: + +1. `GetLatestBlock`: `block {"finality":"optimistic"}` → height, + hash/prev_hash (base58 strings fed to `NewHashIdFromString` as-is). +2. `GetFinalizedBlock`: `block {"finality":"final"}` — implemented for + correctness, but `BlockProcessor()` returns nil in v1, so no separate + finalized-head poll loop runs (same simplification as every non-EVM + family; the optimistic/final gap is ~3 blocks / ~3s). Revisit only if + consumers need a distinct final head. + +### Chain validation + +`NearChainValidator` calls `status` and compares `chain_id` (case-insensitive) +against the configured `chain-id` (`mainnet`/`testnet`/`betanet` — NEAR +chain-ids in `chains.yaml` are network-name strings, not hex). Mismatch → +`FatalSettingError`; fetch error → `SettingsError`. This is exactly +dshackle's check; no genesis call needed (chain_id is unique per network, +unlike bitcoin where two forks both report `main`). + +### Health validation + +`NearHealthValidator` uses the same `status` payload: +- `sync_info.syncing == true` → Syncing; +- stale head guard: `latest_block_time` older than the chain's syncing-lag + worth of block time → Syncing (catches the frozen-head case `syncing:false` + does not); +- peers (when `validate-peers`): `network_info` → `num_active_peers == 0` → + Unavailable. + +### Labels and lower bounds + +- `client_version` label from `status.version.version` (e.g. `2.13.1`), + client type `neard` (the only production implementation). +- `NearLowerBoundDetector`: `status.sync_info.earliest_block_height` → + emitted as STATE and BLOCK bounds (dshackle parity). Our nodes are all + non-archival with a sliding ~5-epoch (~2.5 day) GC window, so the bound is + re-polled on a short period (3 min); no binary search needed — the node + reports its own boundary. + +### Method spec (`pkg/methods/specs/near.json`) + +The 14 methods dshackle serves, all `cacheable: false` in v1: + +| group | methods | notes | +|---|---|---| +| default | block, chunk, tx, EXPERIMENTAL_tx_status, EXPERIMENTAL_receipt, query, EXPERIMENTAL_changes, EXPERIMENTAL_changes_in_block, EXPERIMENTAL_protocol_config, gas_price, validators | object params (`{"finality":...}` / `{"block_id":...}`) | +| passthrough | status, network_info | node-local, never cache | +| broadcast | send_tx | broadcast to all upstreams (dshackle used BroadcastQuorum) | + +No translations, no bans: every upstream is a full neard and serves the whole +surface. Follow-up (not v1): cache tag-parsers for `block`/`chunk`/`tx` by +hash and `query` with explicit `block_id` via the existing jq `tag-parser` +mechanism. + +## Testing + +1. **Unit tests** (bitcoin-family parity): optimistic/final head parsing + (incl. the nanosecond timestamp), chain-id validation mainnet-vs-testnet + cross-check, syncing/stale-head/peers health states, version label, + earliest-block-height bounds, method-spec loading. +2. **Live comparison harness (laptop, before any infra change).** nodecore + runs locally against our production near mainnet + testnet nodes (reached + directly over VPN, plain HTTP, no auth). Corpus: every spec method against + node-direct vs nodecore — blocks by finality/height/hash, deep GC'd + heights (UNKNOWN_BLOCK passthrough), chunk by ids, a real tx and an + unknown tx (408/TIMEOUT_ERROR passthrough within tolerable latency), + query view_account/call_function, changes, protocol/genesis config, + gas_price, validators, unknown method, invalid params. `send_tx` only + with a deterministically invalid transaction on both networks (expected + error passthrough, nothing broadcast). + **Executed 2026-07-17 — passed.** 42/42 checks identical across mainnet + and testnet, byte-identical payloads for all deterministic cases including + GC'd-block UNKNOWN_BLOCK, PARSE_ERROR and the ~10s unknown-tx + TIMEOUT_ERROR passthrough. Expected differences only: nodecore's own + -32601 text for unknown/absent-from-spec methods (local allowlist, + standard for all families), and HTTP transport status on the timeout case + (node replies 408, nodecore returns the identical JSON-RPC error with + 200 — nodecore-wide behavior). Lower bounds verified live: + STATE=BLOCK=the node's earliest_block_height. + +3. **Staged rollout** is deployment-side work, out of scope for this repo. + +## Out of scope / follow-ups + +- Cache tag-parsers and per-method cacheability for finalized data. +- Distinct finalized-head tracking (optimistic vs final) if a consumer needs + it. +- ripple, starknet, ton follow as separate family designs reusing this + template. diff --git a/internal/upstreams/blocks/block_processor.go b/internal/upstreams/blocks/block_processor.go index a1e61a0b..3c938145 100644 --- a/internal/upstreams/blocks/block_processor.go +++ b/internal/upstreams/blocks/block_processor.go @@ -40,31 +40,31 @@ type BlockEvent struct { BlockType protocol.BlockType } -type EthLikeBlockProcessor struct { - upstreamId string - connector connectors.ApiConnector - chainSpecific BlockChainSpecific - subManager *utils.SubscriptionManager[BlockEvent] - disableDetection atomic.Uint32 - disableSafeBlockDetection bool - disableFinalizedBlockDetection bool - manualBlockChan chan *BlockEvent - blocks map[protocol.BlockType]protocol.Block - lifecycle *utils.BaseLifecycle - internalTimeout time.Duration - pollInterval time.Duration -} - -func (b *EthLikeBlockProcessor) Running() bool { +type BaseBlockProcessor struct { + upstreamId string + connector connectors.ApiConnector + chainSpecific BlockChainSpecific + subManager *utils.SubscriptionManager[BlockEvent] + disableDetection atomic.Uint32 + disableSafeBlockDetection bool + disableFinalizedBlockDetection bool + manualBlockChan chan *BlockEvent + blocks map[protocol.BlockType]protocol.Block + lifecycle *utils.BaseLifecycle + internalTimeout time.Duration + pollInterval time.Duration +} + +func (b *BaseBlockProcessor) Running() bool { return b.lifecycle.Running() } -func (b *EthLikeBlockProcessor) Stop() { +func (b *BaseBlockProcessor) Stop() { log.Info().Msgf("stopping block processor of upstream '%s'", b.upstreamId) b.lifecycle.Stop() } -func NewEthLikeBlockProcessor( +func NewBaseBlockProcessor( ctx context.Context, upstreamId string, pollInterval, internalTimeout time.Duration, @@ -72,9 +72,9 @@ func NewEthLikeBlockProcessor( disableSafeBlockDetection bool, connector connectors.ApiConnector, chainSpecific BlockChainSpecific, -) *EthLikeBlockProcessor { +) *BaseBlockProcessor { name := fmt.Sprintf("%s_block_processor", upstreamId) - return &EthLikeBlockProcessor{ + return &BaseBlockProcessor{ upstreamId: upstreamId, connector: connector, chainSpecific: chainSpecific, @@ -89,15 +89,15 @@ func NewEthLikeBlockProcessor( } } -func (b *EthLikeBlockProcessor) UpdateBlock(blockData protocol.Block, blockType protocol.BlockType) { +func (b *BaseBlockProcessor) UpdateBlock(blockData protocol.Block, blockType protocol.BlockType) { b.manualBlockChan <- &BlockEvent{Block: blockData, BlockType: blockType} } -func (b *EthLikeBlockProcessor) Subscribe(name string) *utils.Subscription[BlockEvent] { +func (b *BaseBlockProcessor) Subscribe(name string) *utils.Subscription[BlockEvent] { return b.subManager.Subscribe(name) } -func (b *EthLikeBlockProcessor) Start() { +func (b *BaseBlockProcessor) Start() { b.lifecycle.Start(func(ctx context.Context) error { if !b.disableFinalizedBlockDetection { go b.pollLoop(ctx, protocol.FinalizedBlock) @@ -110,7 +110,7 @@ func (b *EthLikeBlockProcessor) Start() { }) } -func (b *EthLikeBlockProcessor) pollLoop(ctx context.Context, blockType protocol.BlockType) { +func (b *BaseBlockProcessor) pollLoop(ctx context.Context, blockType protocol.BlockType) { b.poll(blockType) ticker := time.NewTicker(b.pollInterval) @@ -126,7 +126,7 @@ func (b *EthLikeBlockProcessor) pollLoop(ctx context.Context, blockType protocol } } -func (b *EthLikeBlockProcessor) blockEventLoop(ctx context.Context) { +func (b *BaseBlockProcessor) blockEventLoop(ctx context.Context) { for { select { case <-ctx.Done(): @@ -141,7 +141,7 @@ func (b *EthLikeBlockProcessor) blockEventLoop(ctx context.Context) { } } -func (b *EthLikeBlockProcessor) poll(blockType protocol.BlockType) { +func (b *BaseBlockProcessor) poll(blockType protocol.BlockType) { if b.detectionDisabled(blockType) { return } @@ -166,11 +166,11 @@ func (b *EthLikeBlockProcessor) poll(blockType protocol.BlockType) { } } -func (b *EthLikeBlockProcessor) detectionDisabled(blockType protocol.BlockType) bool { +func (b *BaseBlockProcessor) detectionDisabled(blockType protocol.BlockType) bool { return b.disableDetection.Load()&blockTypeMask(blockType) != 0 } -func (b *EthLikeBlockProcessor) disableBlockDetection(blockType protocol.BlockType) { +func (b *BaseBlockProcessor) disableBlockDetection(blockType protocol.BlockType) { b.disableDetection.Or(blockTypeMask(blockType)) } @@ -178,7 +178,7 @@ func blockTypeMask(blockType protocol.BlockType) uint32 { return 1 << uint(blockType) } -func (b *EthLikeBlockProcessor) detectBlock(ctx context.Context, blockType protocol.BlockType) (protocol.Block, error) { +func (b *BaseBlockProcessor) detectBlock(ctx context.Context, blockType protocol.BlockType) (protocol.Block, error) { switch blockType { case protocol.FinalizedBlock: return b.chainSpecific.GetFinalizedBlock(ctx) @@ -194,4 +194,4 @@ func (b *EthLikeBlockProcessor) detectBlock(ctx context.Context, blockType proto } } -var _ BlockProcessor = (*EthLikeBlockProcessor)(nil) +var _ BlockProcessor = (*BaseBlockProcessor)(nil) diff --git a/internal/upstreams/blocks/block_processor_test.go b/internal/upstreams/blocks/block_processor_test.go index ac278297..6c1ddc9a 100644 --- a/internal/upstreams/blocks/block_processor_test.go +++ b/internal/upstreams/blocks/block_processor_test.go @@ -19,7 +19,7 @@ import ( "github.com/stretchr/testify/mock" ) -func TestEthLikeBlockProcessorGetFinalizedBlock(t *testing.T) { +func TestBaseBlockProcessorGetFinalizedBlock(t *testing.T) { upConfig := &config.Upstream{Id: "1", PollInterval: 1 * time.Second, Options: &chains.Options{InternalTimeout: 5 * time.Second}} ctx := context.Background() connector := mocks.NewConnectorMock() @@ -35,7 +35,7 @@ func TestEthLikeBlockProcessorGetFinalizedBlock(t *testing.T) { connector.On("SendRequest", mock.Anything, mock.Anything).Return(response) - processor := blocks.NewEthLikeBlockProcessor(ctx, upConfig.Id, upConfig.PollInterval, upConfig.Options.InternalTimeout, false, true, connector, test_utils.NewEvmChainSpecific(connector)) + processor := blocks.NewBaseBlockProcessor(ctx, upConfig.Id, upConfig.PollInterval, upConfig.Options.InternalTimeout, false, true, connector, test_utils.NewEvmChainSpecific(connector)) sub := processor.Subscribe("sub") go processor.Start() @@ -76,7 +76,7 @@ func TestEthLikeBlockProcessorGetFinalizedBlock(t *testing.T) { assert.Equal(t, expected, manualEvent) } -func TestEthLikeBlockProcessorDisableFinalizedBlock(t *testing.T) { +func TestBaseBlockProcessorDisableFinalizedBlock(t *testing.T) { upConfig := &config.Upstream{Id: "1", PollInterval: 1 * time.Second, Options: &chains.Options{InternalTimeout: 5 * time.Second}} ctx := context.Background() connector := mocks.NewConnectorMock() @@ -91,7 +91,7 @@ func TestEthLikeBlockProcessorDisableFinalizedBlock(t *testing.T) { connector.On("SendRequest", mock.Anything, mock.Anything).Return(response) - processor := blocks.NewEthLikeBlockProcessor(ctx, upConfig.Id, upConfig.PollInterval, upConfig.Options.InternalTimeout, false, true, connector, test_utils.NewEvmChainSpecific(connector)) + processor := blocks.NewBaseBlockProcessor(ctx, upConfig.Id, upConfig.PollInterval, upConfig.Options.InternalTimeout, false, true, connector, test_utils.NewEvmChainSpecific(connector)) sub := processor.Subscribe("sub") go processor.Start() @@ -106,14 +106,14 @@ func TestEthLikeBlockProcessorDisableFinalizedBlock(t *testing.T) { assert.False(t, ok) } -func TestEthLikeBlockProcessorPollsSafeBlockWhenSupported(t *testing.T) { +func TestBaseBlockProcessorPollsSafeBlockWhenSupported(t *testing.T) { disableSafe := false upConfig := &config.Upstream{Id: "1", PollInterval: time.Hour, Options: &chains.Options{InternalTimeout: 5 * time.Second, DisableSafeBlockDetection: &disableSafe}} chainSpecific := &blockProcessorChainSpecificStub{ blockProcessorChainSpecificNoSafeStub: blockProcessorChainSpecificNoSafeStub{finalized: protocol.NewBlockWithHeight(100)}, safe: protocol.NewBlockWithHeight(90), } - processor := blocks.NewEthLikeBlockProcessor(context.Background(), upConfig.Id, upConfig.PollInterval, upConfig.Options.InternalTimeout, false, *upConfig.Options.DisableSafeBlockDetection, mocks.NewConnectorMock(), chainSpecific) + processor := blocks.NewBaseBlockProcessor(context.Background(), upConfig.Id, upConfig.PollInterval, upConfig.Options.InternalTimeout, false, *upConfig.Options.DisableSafeBlockDetection, mocks.NewConnectorMock(), chainSpecific) sub := processor.Subscribe("sub") go processor.Start() @@ -133,11 +133,11 @@ func TestEthLikeBlockProcessorPollsSafeBlockWhenSupported(t *testing.T) { assert.Equal(t, uint64(90), seen[protocol.SafeBlock].Height) } -func TestEthLikeBlockProcessorDisablesSafeBlockWhenUnsupported(t *testing.T) { +func TestBaseBlockProcessorDisablesSafeBlockWhenUnsupported(t *testing.T) { disableSafe := false upConfig := &config.Upstream{Id: "1", PollInterval: time.Hour, Options: &chains.Options{InternalTimeout: 5 * time.Second, DisableSafeBlockDetection: &disableSafe}} chainSpecific := &blockProcessorChainSpecificNoSafeStub{finalized: protocol.NewBlockWithHeight(100)} - processor := blocks.NewEthLikeBlockProcessor(context.Background(), upConfig.Id, upConfig.PollInterval, upConfig.Options.InternalTimeout, false, *upConfig.Options.DisableSafeBlockDetection, mocks.NewConnectorMock(), chainSpecific) + processor := blocks.NewBaseBlockProcessor(context.Background(), upConfig.Id, upConfig.PollInterval, upConfig.Options.InternalTimeout, false, *upConfig.Options.DisableSafeBlockDetection, mocks.NewConnectorMock(), chainSpecific) sub := processor.Subscribe("sub") go processor.Start() @@ -205,14 +205,14 @@ func (b *blockProcessorChainSpecificStub) GetSafeBlock(context.Context) (protoco return b.safe, nil } -func TestEthLikeBlockProcessorSkipsSafeBlockWhenDisabled(t *testing.T) { +func TestBaseBlockProcessorSkipsSafeBlockWhenDisabled(t *testing.T) { disableSafe := true upConfig := &config.Upstream{Id: "1", PollInterval: time.Hour, Options: &chains.Options{InternalTimeout: 5 * time.Second, DisableSafeBlockDetection: &disableSafe}} chainSpecific := &blockProcessorChainSpecificStub{ blockProcessorChainSpecificNoSafeStub: blockProcessorChainSpecificNoSafeStub{finalized: protocol.NewBlockWithHeight(100)}, safe: protocol.NewBlockWithHeight(90), } - processor := blocks.NewEthLikeBlockProcessor(context.Background(), upConfig.Id, upConfig.PollInterval, upConfig.Options.InternalTimeout, false, *upConfig.Options.DisableSafeBlockDetection, mocks.NewConnectorMock(), chainSpecific) + processor := blocks.NewBaseBlockProcessor(context.Background(), upConfig.Id, upConfig.PollInterval, upConfig.Options.InternalTimeout, false, *upConfig.Options.DisableSafeBlockDetection, mocks.NewConnectorMock(), chainSpecific) sub := processor.Subscribe("sub") go processor.Start() diff --git a/internal/upstreams/chains_specific/beacon_specific/beacon_chain_specific.go b/internal/upstreams/chains_specific/beacon_specific/beacon_chain_specific.go index 80e9324e..5931c45e 100644 --- a/internal/upstreams/chains_specific/beacon_specific/beacon_chain_specific.go +++ b/internal/upstreams/chains_specific/beacon_specific/beacon_chain_specific.go @@ -64,7 +64,7 @@ func NewBeaconChainSpecificObject( // GetFinalizedBlock (GET /eth/v1/beacon/headers/finalized). Safe-block detection // is disabled - beacon has no "safe" head and doesn't implement GetSafeBlock. func (b *BeaconChainSpecificObject) BlockProcessor() blocks.BlockProcessor { - return blocks.NewEthLikeBlockProcessor( + return blocks.NewBaseBlockProcessor( b.ctx, b.upstreamId, b.pollInterval, diff --git a/internal/upstreams/chains_specific/evm_specific/evm_chain_specific.go b/internal/upstreams/chains_specific/evm_specific/evm_chain_specific.go index 4c9d1db9..5a2405eb 100644 --- a/internal/upstreams/chains_specific/evm_specific/evm_chain_specific.go +++ b/internal/upstreams/chains_specific/evm_specific/evm_chain_specific.go @@ -36,7 +36,7 @@ type EvmChainSpecificObject struct { } func (e *EvmChainSpecificObject) BlockProcessor() blocks.BlockProcessor { - return blocks.NewEthLikeBlockProcessor( + return blocks.NewBaseBlockProcessor( e.ctx, e.upstreamId, e.pollInterval, diff --git a/internal/upstreams/chains_specific/near_specific/near_chain_specific.go b/internal/upstreams/chains_specific/near_specific/near_chain_specific.go new file mode 100644 index 00000000..3bc275c3 --- /dev/null +++ b/internal/upstreams/chains_specific/near_specific/near_chain_specific.go @@ -0,0 +1,213 @@ +package near_specific + +import ( + "context" + "fmt" + "time" + + "github.com/bytedance/sonic" + "github.com/drpcorg/nodecore/internal/protocol" + "github.com/drpcorg/nodecore/internal/upstreams/blocks" + "github.com/drpcorg/nodecore/internal/upstreams/caps" + "github.com/drpcorg/nodecore/internal/upstreams/chains_specific" + "github.com/drpcorg/nodecore/internal/upstreams/connectors" + "github.com/drpcorg/nodecore/internal/upstreams/labels" + "github.com/drpcorg/nodecore/internal/upstreams/labels/near_labels" + "github.com/drpcorg/nodecore/internal/upstreams/lower_bounds" + "github.com/drpcorg/nodecore/internal/upstreams/lower_bounds/near_bounds" + "github.com/drpcorg/nodecore/internal/upstreams/validations" + "github.com/drpcorg/nodecore/internal/upstreams/validations/near_validations" + "github.com/drpcorg/nodecore/pkg/blockchain" + "github.com/drpcorg/nodecore/pkg/chains" +) + +// errUnsupportedHeadSubscriptions is returned by SubscribeHeadRequest and +// ParseSubscriptionBlock: nearcore's JSON-RPC is HTTP request/response only, +// there is no subscription transport at all, so head tracking is poll-only. +var errUnsupportedHeadSubscriptions = fmt.Errorf("near: head subscriptions are not supported") + +type NearChainSpecificObject struct { + ctx context.Context + upstreamId string + connector connectors.ApiConnector + options *chains.Options + pollInterval time.Duration + internalTimeout time.Duration + labelsDelay time.Duration + configuredChain *chains.ConfiguredChain +} + +func NewNearChainSpecificObject( + ctx context.Context, + configuredChain *chains.ConfiguredChain, + upstreamId string, + connector connectors.ApiConnector, + pollInterval time.Duration, + options *chains.Options, +) *NearChainSpecificObject { + return &NearChainSpecificObject{ + ctx: ctx, + upstreamId: upstreamId, + connector: connector, + options: options, + pollInterval: pollInterval, + internalTimeout: options.InternalTimeout, + labelsDelay: options.ValidationInterval * 5, + configuredChain: configuredChain, + } +} + +func (n *NearChainSpecificObject) BlockProcessor() blocks.BlockProcessor { + return blocks.NewBaseBlockProcessor( + n.ctx, + n.upstreamId, + n.pollInterval, + n.internalTimeout, + n.options.FinalizedBlockDetectionDisabled(), + true, // near has no "safe" block concept, only optimistic/final + n.connector, + n, + ) +} + +func (n *NearChainSpecificObject) LabelsProcessor() labels.LabelsProcessor { + labelsDetectors := []labels.LabelsDetector{ + labels.NewClientLabelDetectorHandler( + n.upstreamId, + n.connector, + near_labels.NewNearClientLabelsDetector(n.configuredChain.Chain), + n.internalTimeout, + ), + } + return labels.NewBaseLabelsProcessor(n.ctx, n.upstreamId, labelsDetectors, n.labelsDelay) +} + +func (n *NearChainSpecificObject) CapDetectors(_ caps.DetectorInput) []caps.CapDetector { + // nearcore has no websocket transport, so no ws-derived caps can ever be asserted + return nil +} + +func (n *NearChainSpecificObject) LowerBoundProcessor() lower_bounds.LowerBoundProcessor { + detectors := []lower_bounds.LowerBoundDetector{ + near_bounds.NewNearLowerBoundDetector( + n.upstreamId, + n.configuredChain.Chain, + n.internalTimeout, + n.connector, + ), + } + return lower_bounds.NewBaseLowerBoundProcessor( + n.ctx, + n.upstreamId, + n.configuredChain.AverageRemoveSpeed(), + detectors, + ) +} + +func (n *NearChainSpecificObject) HealthValidators() []validations.Validator[protocol.AvailabilityStatus] { + if n.options != nil && *n.options.DisableHealthValidation { + return []validations.Validator[protocol.AvailabilityStatus]{} + } + validators := make([]validations.Validator[protocol.AvailabilityStatus], 0) + if n.options != nil && n.options.ValidateSyncing != nil && *n.options.ValidateSyncing { + validators = append(validators, near_validations.NewNearSyncingValidator( + n.upstreamId, n.connector, n.configuredChain, n.internalTimeout, + )) + } + if n.options != nil && n.options.ValidatePeers != nil && *n.options.ValidatePeers { + validators = append(validators, near_validations.NewNearPeersValidator( + n.upstreamId, n.connector, n.configuredChain, n.internalTimeout, + )) + } + return validators +} + +func (n *NearChainSpecificObject) SettingsValidators() []validations.Validator[validations.ValidationSettingResult] { + if n.configuredChain == nil || n.configuredChain.ChainId == "" { + return nil + } + if n.options != nil && *n.options.DisableChainValidation { + return []validations.Validator[validations.ValidationSettingResult]{} + } + return []validations.Validator[validations.ValidationSettingResult]{ + near_validations.NewNearChainValidator(n.upstreamId, n.connector, n.configuredChain, n.internalTimeout), + } +} + +// GetLatestBlock polls the optimistic head - the same head dshackle exposed +// for near, ~3 blocks ahead of "final". +func (n *NearChainSpecificObject) GetLatestBlock(ctx context.Context) (protocol.Block, error) { + return n.fetchBlockByFinality(ctx, "optimistic") +} + +// GetFinalizedBlock fetches the "final" head; the block processor polls it +// alongside the optimistic head tracked by the head processor. +func (n *NearChainSpecificObject) GetFinalizedBlock(ctx context.Context) (protocol.Block, error) { + return n.fetchBlockByFinality(ctx, "final") +} + +func (n *NearChainSpecificObject) fetchBlockByFinality(ctx context.Context, finality string) (protocol.Block, error) { + request, err := protocol.NewInternalUpstreamJsonRpcRequest( + "block", + map[string]any{"finality": finality}, + n.configuredChain.Chain, + ) + if err != nil { + return protocol.ZeroBlock{}, err + } + response := n.connector.SendRequest(ctx, request) + if response.HasError() { + return protocol.ZeroBlock{}, response.GetError() + } + + block, err := n.ParseBlock(response.ResponseResult()) + if err != nil { + return protocol.ZeroBlock{}, fmt.Errorf("couldn't parse the near %s block: %w", finality, err) + } + return block, nil +} + +// ParseBlock expects the payload shape of the `block` method: +// {"author": ..., "header": {"height", "hash", "prev_hash", ...}, "chunks": [...]}. +// Hashes are base58 strings and are used verbatim as block identifiers. +func (n *NearChainSpecificObject) ParseBlock(blockBytes []byte) (protocol.Block, error) { + block := nearBlock{} + if err := sonic.Unmarshal(blockBytes, &block); err != nil { + return protocol.ZeroBlock{}, fmt.Errorf("couldn't parse the near block, reason - %s", err.Error()) + } + if block.Header.Hash == "" { + return protocol.ZeroBlock{}, fmt.Errorf("couldn't parse the near block, got '%s'", string(blockBytes)) + } + + parentHash := blockchain.EmptyHash + if block.Header.PrevHash != "" { + parentHash = blockchain.NewHashIdFromString(block.Header.PrevHash) + } + + return protocol.NewBlock( + block.Header.Height, + 0, + blockchain.NewHashIdFromString(block.Header.Hash), + parentHash, + ), nil +} + +func (n *NearChainSpecificObject) ParseSubscriptionBlock(_ []byte) (protocol.Block, error) { + return protocol.ZeroBlock{}, errUnsupportedHeadSubscriptions +} + +func (n *NearChainSpecificObject) SubscribeHeadRequest() (protocol.RequestHolder, error) { + return nil, errUnsupportedHeadSubscriptions +} + +type nearBlock struct { + Header nearBlockHeader `json:"header"` +} + +type nearBlockHeader struct { + Height uint64 `json:"height"` + Hash string `json:"hash"` + PrevHash string `json:"prev_hash"` +} + +var _ chains_specific.ChainSpecific = (*NearChainSpecificObject)(nil) diff --git a/internal/upstreams/chains_specific/near_specific/near_chain_specific_test.go b/internal/upstreams/chains_specific/near_specific/near_chain_specific_test.go new file mode 100644 index 00000000..cdcd723b --- /dev/null +++ b/internal/upstreams/chains_specific/near_specific/near_chain_specific_test.go @@ -0,0 +1,124 @@ +package near_specific_test + +import ( + "context" + "strings" + "testing" + + "github.com/drpcorg/nodecore/internal/protocol" + "github.com/drpcorg/nodecore/internal/upstreams/caps" + "github.com/drpcorg/nodecore/pkg/blockchain" + "github.com/drpcorg/nodecore/pkg/test_utils" + "github.com/drpcorg/nodecore/pkg/test_utils/mocks" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/mock" + "github.com/stretchr/testify/require" +) + +func TestNearSubscribeHeadRequest(t *testing.T) { + req, err := test_utils.NewNearChainSpecific(context.Background(), nil).SubscribeHeadRequest() + assert.Nil(t, req) + assert.EqualError(t, err, "near: head subscriptions are not supported") +} + +func TestNearParseSubscriptionBlock(t *testing.T) { + block, err := test_utils.NewNearChainSpecific(context.Background(), nil).ParseSubscriptionBlock([]byte(`{}`)) + assert.True(t, block.IsFullEmpty()) + assert.EqualError(t, err, "near: head subscriptions are not supported") +} + +func TestNearParseBlock(t *testing.T) { + body := []byte(`{ + "author": "validator.near", + "header": {"height": 207365026, "hash": "9nEcHpjcsfjMwHzHYzDLZeLBEbeqbNRew7oXCSFvi2Wa", "prev_hash": "5qJoxdRBSDaZmGuLzjWfDGnWqCvHdRPuJqkyZv7QwXvJ"}, + "chunks": [] + }`) + + block, err := test_utils.NewNearChainSpecific(context.Background(), nil).ParseBlock(body) + require.NoError(t, err) + + expected := protocol.NewBlock( + 207365026, + 0, + blockchain.NewHashIdFromString("9nEcHpjcsfjMwHzHYzDLZeLBEbeqbNRew7oXCSFvi2Wa"), + blockchain.NewHashIdFromString("5qJoxdRBSDaZmGuLzjWfDGnWqCvHdRPuJqkyZv7QwXvJ"), + ) + assert.Equal(t, expected, block) +} + +func TestNearParseBlockNoPrevHash(t *testing.T) { + body := []byte(`{"header": {"height": 1, "hash": "9nEcHpjcsfjMwHzHYzDLZeLBEbeqbNRew7oXCSFvi2Wa"}}`) + + block, err := test_utils.NewNearChainSpecific(context.Background(), nil).ParseBlock(body) + require.NoError(t, err) + assert.Equal(t, blockchain.EmptyHash, block.ParentHash) +} + +func TestNearParseBlockInvalidJSON(t *testing.T) { + block, err := test_utils.NewNearChainSpecific(context.Background(), nil).ParseBlock([]byte(`not json`)) + assert.True(t, block.IsFullEmpty()) + assert.ErrorContains(t, err, "couldn't parse the near block") +} + +func TestNearParseBlockNoHash(t *testing.T) { + block, err := test_utils.NewNearChainSpecific(context.Background(), nil).ParseBlock([]byte(`{"header":{"height":1}}`)) + assert.True(t, block.IsFullEmpty()) + assert.ErrorContains(t, err, "couldn't parse the near block") +} + +func matchBlockRequestWithFinality(finality string) func(protocol.RequestHolder) bool { + return func(req protocol.RequestHolder) bool { + if req.Method() != "block" { + return false + } + body, err := req.Body() + if err != nil { + return false + } + return strings.Contains(string(body), `"finality":"`+finality+`"`) + } +} + +func TestNearGetLatestBlockPollsOptimistic(t *testing.T) { + connector := mocks.NewConnectorMock() + body := []byte(`{"header": {"height": 207365026, "hash": "9nEcHpjcsfjMwHzHYzDLZeLBEbeqbNRew7oXCSFvi2Wa", "prev_hash": "5qJoxdRBSDaZmGuLzjWfDGnWqCvHdRPuJqkyZv7QwXvJ"}}`) + connector.On("SendRequest", mock.Anything, mock.MatchedBy(matchBlockRequestWithFinality("optimistic"))). + Return(protocol.NewSimpleHttpUpstreamResponse("1", body, protocol.JsonRpc)) + + block, err := test_utils.NewNearChainSpecific(context.Background(), connector).GetLatestBlock(context.Background()) + require.NoError(t, err) + assert.Equal(t, uint64(207365026), block.Height) + connector.AssertExpectations(t) +} + +func TestNearGetFinalizedBlockPollsFinal(t *testing.T) { + connector := mocks.NewConnectorMock() + body := []byte(`{"header": {"height": 207365020, "hash": "5qJoxdRBSDaZmGuLzjWfDGnWqCvHdRPuJqkyZv7QwXvJ", "prev_hash": "9nEcHpjcsfjMwHzHYzDLZeLBEbeqbNRew7oXCSFvi2Wa"}}`) + connector.On("SendRequest", mock.Anything, mock.MatchedBy(matchBlockRequestWithFinality("final"))). + Return(protocol.NewSimpleHttpUpstreamResponse("1", body, protocol.JsonRpc)) + + block, err := test_utils.NewNearChainSpecific(context.Background(), connector).GetFinalizedBlock(context.Background()) + require.NoError(t, err) + assert.Equal(t, uint64(207365020), block.Height) + connector.AssertExpectations(t) +} + +func TestNearGetLatestBlockError(t *testing.T) { + connector := mocks.NewConnectorMock() + connector.On("SendRequest", mock.Anything, mock.Anything). + Return(protocol.NewHttpUpstreamResponseWithError(protocol.ResponseErrorWithData(1, "boom", nil))) + + block, err := test_utils.NewNearChainSpecific(context.Background(), connector).GetLatestBlock(context.Background()) + assert.True(t, block.IsFullEmpty()) + assert.Error(t, err) +} + +func TestNearBlockProcessorIsCreated(t *testing.T) { + processor := test_utils.NewNearChainSpecific(context.Background(), mocks.NewConnectorMock()).BlockProcessor() + assert.NotNil(t, processor) +} + +func TestNearCapDetectorsAreNil(t *testing.T) { + detectors := test_utils.NewNearChainSpecific(context.Background(), nil).CapDetectors(caps.DetectorInput{}) + assert.Nil(t, detectors) +} diff --git a/internal/upstreams/chains_specific/tron_specific/tron_rest_specific.go b/internal/upstreams/chains_specific/tron_specific/tron_rest_specific.go index c0c80b69..2ea48f7a 100644 --- a/internal/upstreams/chains_specific/tron_specific/tron_rest_specific.go +++ b/internal/upstreams/chains_specific/tron_specific/tron_rest_specific.go @@ -35,7 +35,7 @@ type TronRestSpecific struct { } func (t *TronRestSpecific) BlockProcessor() blocks.BlockProcessor { - return blocks.NewEthLikeBlockProcessor( + return blocks.NewBaseBlockProcessor( t.ctx, t.upstreamId, t.pollInterval, diff --git a/internal/upstreams/labels/near_labels/near_detectors.go b/internal/upstreams/labels/near_labels/near_detectors.go new file mode 100644 index 00000000..c4a60153 --- /dev/null +++ b/internal/upstreams/labels/near_labels/near_detectors.go @@ -0,0 +1,42 @@ +package near_labels + +import ( + "fmt" + + "github.com/bytedance/sonic" + "github.com/drpcorg/nodecore/internal/protocol" + "github.com/drpcorg/nodecore/internal/upstreams/labels" + "github.com/drpcorg/nodecore/pkg/chains" +) + +// neard is the only production NEAR client, so the type is a constant +const nearClientType = "neard" + +type NearClientLabelsDetector struct { + chain chains.Chain +} + +func NewNearClientLabelsDetector(chain chains.Chain) *NearClientLabelsDetector { + return &NearClientLabelsDetector{chain: chain} +} + +func (n *NearClientLabelsDetector) NodeTypeRequest() (protocol.RequestHolder, error) { + return protocol.NewInternalUpstreamJsonRpcRequest("status", []any{}, n.chain) +} + +type nearStatusResponse struct { + Version struct { + Version string `json:"version"` + } `json:"version"` +} + +func (n *NearClientLabelsDetector) ClientVersionAndType(data []byte) (string, string, error) { + var status nearStatusResponse + if err := sonic.Unmarshal(data, &status); err != nil { + return "", "", fmt.Errorf("near status payload unparseable: %w", err) + } + // version looks like {"version":{"version":"2.13.1","build":"unknown","rustc_version":"1.93.0"},...} + return status.Version.Version, nearClientType, nil +} + +var _ labels.ClientLabelsDetector = (*NearClientLabelsDetector)(nil) diff --git a/internal/upstreams/labels/near_labels/near_detectors_test.go b/internal/upstreams/labels/near_labels/near_detectors_test.go new file mode 100644 index 00000000..93e0846b --- /dev/null +++ b/internal/upstreams/labels/near_labels/near_detectors_test.go @@ -0,0 +1,80 @@ +package near_labels_test + +import ( + "testing" + + "github.com/drpcorg/nodecore/internal/protocol" + "github.com/drpcorg/nodecore/internal/upstreams/labels/near_labels" + "github.com/drpcorg/nodecore/pkg/chains" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/require" +) + +func TestNearClientLabelsDetectorNodeTypeRequest(t *testing.T) { + detector := near_labels.NewNearClientLabelsDetector(chains.NEAR) + + request, err := detector.NodeTypeRequest() + require.NoError(t, err) + require.NotNil(t, request) + + assert.Equal(t, "status", request.Method()) + assert.Equal(t, protocol.JsonRpc, request.RequestType()) + + body, err := request.Body() + require.NoError(t, err) + + assert.JSONEq(t, `{"id":"1","jsonrpc":"2.0","method":"status","params":[]}`, string(body)) +} + +func TestNearClientLabelsDetectorClientVersionAndType(t *testing.T) { + tests := []struct { + name string + data []byte + expectedVersion string + expectedClientType string + expectErr bool + }{ + { + name: "parses neard version", + data: []byte(`{"version":{"version":"2.13.1","build":"unknown","rustc_version":"1.93.0"},"chain_id":"mainnet"}`), + expectedVersion: "2.13.1", + expectedClientType: "neard", + }, + { + name: "falls back on missing version", + data: []byte(`{"chain_id":"mainnet"}`), + expectedVersion: "", + expectedClientType: "neard", + }, + { + name: "falls back on empty version", + data: []byte(`{"version":{"version":"","build":"unknown"}}`), + expectedVersion: "", + expectedClientType: "neard", + }, + { + name: "fails on invalid json", + data: []byte(`{"version":`), + expectErr: true, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + detector := near_labels.NewNearClientLabelsDetector(chains.NEAR) + + version, clientType, err := detector.ClientVersionAndType(tt.data) + + if tt.expectErr { + require.Error(t, err) + assert.Empty(t, version) + assert.Empty(t, clientType) + return + } + + require.NoError(t, err) + assert.Equal(t, tt.expectedVersion, version) + assert.Equal(t, tt.expectedClientType, clientType) + }) + } +} diff --git a/internal/upstreams/lower_bounds/near_bounds/near_lower_bound.go b/internal/upstreams/lower_bounds/near_bounds/near_lower_bound.go new file mode 100644 index 00000000..bd280ada --- /dev/null +++ b/internal/upstreams/lower_bounds/near_bounds/near_lower_bound.go @@ -0,0 +1,121 @@ +package near_bounds + +import ( + "context" + "errors" + "fmt" + "time" + + "github.com/bytedance/sonic" + "github.com/drpcorg/nodecore/internal/protocol" + "github.com/drpcorg/nodecore/internal/upstreams/connectors" + "github.com/drpcorg/nodecore/internal/upstreams/lower_bounds" + "github.com/drpcorg/nodecore/pkg/chains" + "github.com/failsafe-go/failsafe-go" + "github.com/failsafe-go/failsafe-go/retrypolicy" +) + +// Non-archival nodes garbage-collect a sliding ~5-epoch window, so the +// earliest available block moves at roughly one block per second; re-poll +// often to keep the bound close to the real GC boundary. +const nearPeriod = 3 * time.Minute + +const ( + nearRetryAttempts = 3 + nearRetryDelay = 500 * time.Millisecond +) + +var errNearNoEarliestHeight = errors.New("near node reported no earliest_block_height") + +// NearLowerBoundDetector reads earliest_block_height from the `status` +// response - nearcore publishes its GC boundary directly, one RPC per +// refresh. On any error the detector returns (nil, err): the processor +// logs it, skips the tick, and the previously cached bound stays in place. +type NearLowerBoundDetector struct { + upstreamId string + connector connectors.ApiConnector + chain chains.Chain + internalTimeout time.Duration +} + +func NewNearLowerBoundDetector( + upstreamId string, + chain chains.Chain, + internalTimeout time.Duration, + connector connectors.ApiConnector, +) *NearLowerBoundDetector { + return &NearLowerBoundDetector{ + upstreamId: upstreamId, + connector: connector, + chain: chain, + internalTimeout: internalTimeout, + } +} + +func (n *NearLowerBoundDetector) DetectLowerBound(ctx context.Context) ([]protocol.LowerBoundData, error) { + retryPolicy := retrypolicy.NewBuilder[*nearStatus](). + WithMaxAttempts(nearRetryAttempts). + WithDelay(nearRetryDelay). + ReturnLastFailure(). + Build() + status, err := failsafe.With(retryPolicy).WithContext(ctx).Get(func() (*nearStatus, error) { + return n.fetchStatus(ctx) + }) + if err != nil { + return nil, fmt.Errorf("cannot fetch near status for upstream '%s': %w", n.upstreamId, err) + } + + bound := status.SyncInfo.EarliestBlockHeight + if bound <= 0 { + // Zero or absent means the node did not report its GC boundary, + // not that the full history is available. + return nil, errNearNoEarliestHeight + } + + return []protocol.LowerBoundData{ + protocol.NewLowerBoundDataNow(bound, protocol.StateBound), + protocol.NewLowerBoundDataNow(bound, protocol.BlockBound), + }, nil +} + +func (n *NearLowerBoundDetector) SupportedTypes() []protocol.LowerBoundType { + return []protocol.LowerBoundType{protocol.StateBound, protocol.BlockBound} +} + +func (n *NearLowerBoundDetector) Period() time.Duration { + return nearPeriod +} + +func (n *NearLowerBoundDetector) fetchStatus(ctx context.Context) (*nearStatus, error) { + ctx, cancel := context.WithTimeout(ctx, n.internalTimeout) + defer cancel() + + request, err := protocol.NewInternalUpstreamJsonRpcRequest("status", []any{}, n.chain) + if err != nil { + return nil, err + } + + response := n.connector.SendRequest(ctx, request) + if response.HasError() { + return nil, response.GetError() + } + raw := response.ResponseResult() + if len(raw) == 0 { + return nil, fmt.Errorf("near upstream '%s' status returned an empty body", n.upstreamId) + } + var status nearStatus + if err := sonic.Unmarshal(raw, &status); err != nil { + return nil, fmt.Errorf("near upstream '%s' status unparseable: %w", n.upstreamId, err) + } + return &status, nil +} + +type nearStatus struct { + SyncInfo nearSyncInfo `json:"sync_info"` +} + +type nearSyncInfo struct { + EarliestBlockHeight int64 `json:"earliest_block_height"` +} + +var _ lower_bounds.LowerBoundDetector = (*NearLowerBoundDetector)(nil) diff --git a/internal/upstreams/lower_bounds/near_bounds/near_lower_bound_test.go b/internal/upstreams/lower_bounds/near_bounds/near_lower_bound_test.go new file mode 100644 index 00000000..0eca2d00 --- /dev/null +++ b/internal/upstreams/lower_bounds/near_bounds/near_lower_bound_test.go @@ -0,0 +1,133 @@ +package near_bounds_test + +import ( + "context" + "fmt" + "testing" + "time" + + "github.com/drpcorg/nodecore/internal/protocol" + "github.com/drpcorg/nodecore/internal/upstreams/lower_bounds/near_bounds" + "github.com/drpcorg/nodecore/pkg/chains" + "github.com/drpcorg/nodecore/pkg/test_utils/mocks" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/mock" + "github.com/stretchr/testify/require" +) + +func nearStatusResponse(earliestBlockHeight int64) protocol.ResponseHolder { + body := fmt.Sprintf( + `{"jsonrpc":"2.0","id":"1","result":{"chain_id":"mainnet","sync_info":{"latest_block_height":207365026,"earliest_block_height":%d,"syncing":false}}}`, + earliestBlockHeight, + ) + return protocol.NewHttpUpstreamResponse("1", []byte(body), 200, protocol.JsonRpc) +} + +func nearStatusResponseWithoutEarliest() protocol.ResponseHolder { + body := `{"jsonrpc":"2.0","id":"1","result":{"chain_id":"mainnet","sync_info":{"latest_block_height":207365026,"syncing":false}}}` + return protocol.NewHttpUpstreamResponse("1", []byte(body), 200, protocol.JsonRpc) +} + +func matchNearStatusRequest() func(protocol.RequestHolder) bool { + return func(req protocol.RequestHolder) bool { + return req.Method() == "status" && req.RequestType() == protocol.JsonRpc + } +} + +func boundsByType(t *testing.T, result []protocol.LowerBoundData) map[protocol.LowerBoundType]int64 { + t.Helper() + got := make(map[protocol.LowerBoundType]int64, len(result)) + for _, b := range result { + got[b.Type] = b.Bound + } + return got +} + +func TestNearLowerBoundDetector_SupportedTypesAndPeriod(t *testing.T) { + connector := mocks.NewConnectorMock() + detector := near_bounds.NewNearLowerBoundDetector("id", chains.NEAR, time.Second, connector) + + assert.ElementsMatch(t, + []protocol.LowerBoundType{ + protocol.StateBound, + protocol.BlockBound, + }, + detector.SupportedTypes(), + ) + assert.Equal(t, 3*time.Minute, detector.Period()) +} + +func TestNearLowerBoundDetector_EarliestHeightReturnsStateAndBlockBounds(t *testing.T) { + connector := mocks.NewConnectorMock() + connector. + On("SendRequest", mock.Anything, mock.MatchedBy(matchNearStatusRequest())). + Return(nearStatusResponse(207157933)) + + detector := near_bounds.NewNearLowerBoundDetector("id", chains.NEAR, time.Second, connector) + + result, err := detector.DetectLowerBound(context.Background()) + require.NoError(t, err) + require.Len(t, result, 2) + got := boundsByType(t, result) + assert.Equal(t, int64(207157933), got[protocol.StateBound]) + assert.Equal(t, int64(207157933), got[protocol.BlockBound]) +} + +func TestNearLowerBoundDetector_RetriesTransientErrorThenSucceeds(t *testing.T) { + connector := mocks.NewConnectorMock() + connector. + On("SendRequest", mock.Anything, mock.MatchedBy(matchNearStatusRequest())). + Return(protocol.NewHttpUpstreamResponseWithError(protocol.ServerError())). + Once() + connector. + On("SendRequest", mock.Anything, mock.MatchedBy(matchNearStatusRequest())). + Return(nearStatusResponse(207157933)) + + detector := near_bounds.NewNearLowerBoundDetector("id", chains.NEAR, time.Second, connector) + + result, err := detector.DetectLowerBound(context.Background()) + require.NoError(t, err) + require.Len(t, result, 2) + got := boundsByType(t, result) + assert.Equal(t, int64(207157933), got[protocol.StateBound]) + assert.Equal(t, int64(207157933), got[protocol.BlockBound]) +} + +func TestNearLowerBoundDetector_ErrorReturnsError(t *testing.T) { + connector := mocks.NewConnectorMock() + connector. + On("SendRequest", mock.Anything, mock.MatchedBy(matchNearStatusRequest())). + Return(protocol.NewHttpUpstreamResponseWithError(protocol.ServerError())) + + detector := near_bounds.NewNearLowerBoundDetector("id", chains.NEAR, time.Second, connector) + + result, err := detector.DetectLowerBound(context.Background()) + require.Error(t, err) + assert.Nil(t, result) +} + +func TestNearLowerBoundDetector_ZeroEarliestHeightReturnsError(t *testing.T) { + connector := mocks.NewConnectorMock() + connector. + On("SendRequest", mock.Anything, mock.MatchedBy(matchNearStatusRequest())). + Return(nearStatusResponse(0)) + + detector := near_bounds.NewNearLowerBoundDetector("id", chains.NEAR, time.Second, connector) + + result, err := detector.DetectLowerBound(context.Background()) + require.ErrorContains(t, err, "no earliest_block_height") + assert.Nil(t, result) +} + +func TestNearLowerBoundDetector_AbsentEarliestHeightReturnsError(t *testing.T) { + connector := mocks.NewConnectorMock() + connector. + On("SendRequest", mock.Anything, mock.MatchedBy(matchNearStatusRequest())). + Return(nearStatusResponseWithoutEarliest()) + + detector := near_bounds.NewNearLowerBoundDetector("id", chains.NEAR, time.Second, connector) + + result, err := detector.DetectLowerBound(context.Background()) + require.ErrorContains(t, err, "no earliest_block_height") + assert.Nil(t, result) +} diff --git a/internal/upstreams/upstream_factory.go b/internal/upstreams/upstream_factory.go index 147d019b..452e7905 100644 --- a/internal/upstreams/upstream_factory.go +++ b/internal/upstreams/upstream_factory.go @@ -12,6 +12,7 @@ import ( "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/beacon_specific" "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/bitcoin_specific" "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/evm_specific" + "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/near_specific" "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/solana_specific" "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/tron_specific" "github.com/drpcorg/nodecore/pkg/methods" @@ -252,6 +253,15 @@ func getChainSpecific( upstreamConnectorsInfo.internalRequestConnector, conf.Options, ), nil + case chains.Near: + return near_specific.NewNearChainSpecificObject( + ctx, + configuredChain, + conf.Id, + upstreamConnectorsInfo.internalRequestConnector, + conf.PollInterval, + conf.Options, + ), nil case chains.Solana: return solana_specific.NewSolanaChainSpecificObject( ctx, diff --git a/internal/upstreams/validations/near_validations/near_chain_validator.go b/internal/upstreams/validations/near_validations/near_chain_validator.go new file mode 100644 index 00000000..38e81577 --- /dev/null +++ b/internal/upstreams/validations/near_validations/near_chain_validator.go @@ -0,0 +1,58 @@ +package near_validations + +import ( + "errors" + "strings" + "time" + + "github.com/drpcorg/nodecore/internal/upstreams/connectors" + "github.com/drpcorg/nodecore/internal/upstreams/validations" + "github.com/drpcorg/nodecore/pkg/chains" + "github.com/rs/zerolog/log" +) + +var errNearEmptyChainId = errors.New("near node returned empty chain_id") + +type NearChainValidator struct { + nearStatusClient + upstreamId string +} + +func NewNearChainValidator( + upstreamId string, + connector connectors.ApiConnector, + chain *chains.ConfiguredChain, + internalTimeout time.Duration, +) *NearChainValidator { + return &NearChainValidator{ + nearStatusClient: nearStatusClient{connector: connector, chain: chain, internalTimeout: internalTimeout}, + upstreamId: upstreamId, + } +} + +func (n *NearChainValidator) Validate() validations.ValidationSettingResult { + status, err := n.fetchNearStatus() + if err != nil { + log.Error().Err(err).Msgf("failed to fetch near status for upstream '%s'", n.upstreamId) + return validations.SettingsError + } + if status.ChainId == "" { + // no chain_id means we can't tell what network the node is on - unusable as configured + log.Error().Err(errNearEmptyChainId).Msgf("failed to validate the chain of near upstream '%s'", n.upstreamId) + return validations.FatalSettingError + } + // for near chains.yaml holds network-name chain-ids: mainnet/testnet/betanet + if strings.EqualFold(status.ChainId, n.chain.ChainId) { + return validations.Valid + } + log.Error().Msgf( + "'%s' expects chain_id '%s' but near upstream '%s' reports '%s'", + n.chain.Chain.String(), + n.chain.ChainId, + n.upstreamId, + status.ChainId, + ) + return validations.FatalSettingError +} + +var _ validations.SettingsValidator = (*NearChainValidator)(nil) diff --git a/internal/upstreams/validations/near_validations/near_health_validator.go b/internal/upstreams/validations/near_validations/near_health_validator.go new file mode 100644 index 00000000..0c8b34e3 --- /dev/null +++ b/internal/upstreams/validations/near_validations/near_health_validator.go @@ -0,0 +1,106 @@ +package near_validations + +import ( + "context" + "time" + + "github.com/bytedance/sonic" + "github.com/drpcorg/nodecore/internal/protocol" + "github.com/drpcorg/nodecore/internal/upstreams/connectors" + "github.com/drpcorg/nodecore/internal/upstreams/validations" + "github.com/drpcorg/nodecore/pkg/chains" + "github.com/rs/zerolog/log" +) + +// nearStatusClient holds what a `status` call needs; validators embed it and +// call fetchNearStatus without passing params around. +type nearStatusClient struct { + connector connectors.ApiConnector + chain *chains.ConfiguredChain + internalTimeout time.Duration +} + +func (n *nearStatusClient) fetchNearStatus() (*NearStatus, error) { + ctx, cancel := context.WithTimeout(context.Background(), n.internalTimeout) + defer cancel() + + request, err := protocol.NewInternalUpstreamJsonRpcRequest("status", []any{}, n.chain.Chain) + if err != nil { + return nil, err + } + response := n.connector.SendRequest(ctx, request) + if response.HasError() { + return nil, response.GetError() + } + var status NearStatus + if err := sonic.Unmarshal(response.ResponseResult(), &status); err != nil { + return nil, err + } + return &status, nil +} + +// NearSyncingValidator checks the node's own sync state via `status`: +// the explicit syncing flag plus a stale-head guard. +type NearSyncingValidator struct { + nearStatusClient + upstreamId string +} + +func NewNearSyncingValidator( + upstreamId string, + connector connectors.ApiConnector, + chain *chains.ConfiguredChain, + internalTimeout time.Duration, +) *NearSyncingValidator { + return &NearSyncingValidator{ + nearStatusClient: nearStatusClient{connector: connector, chain: chain, internalTimeout: internalTimeout}, + upstreamId: upstreamId, + } +} + +func (n *NearSyncingValidator) Validate() protocol.AvailabilityStatus { + status, err := n.fetchNearStatus() + if err != nil { + log.Error().Err(err).Msgf("near upstream '%s' syncing validation failed", n.upstreamId) + return protocol.Unavailable + } + if status.SyncInfo.Syncing { + log.Warn().Msgf("near upstream '%s' is syncing", n.upstreamId) + return protocol.Syncing + } + // stale-head guard: syncing=false with a frozen head is still not serviceable + maxHeadAge := n.chain.Settings.ExpectedBlockTime * time.Duration(n.chain.Settings.Lags.Syncing) + if maxHeadAge > 0 { + if latestBlockTime, parseErr := time.Parse(time.RFC3339Nano, status.SyncInfo.LatestBlockTime); parseErr == nil { + if headAge := time.Since(latestBlockTime); headAge > maxHeadAge { + log.Warn().Msgf( + "near upstream '%s' head is stale, latest_block_time=%s age=%s", + n.upstreamId, + status.SyncInfo.LatestBlockTime, + headAge, + ) + return protocol.Syncing + } + } + } + return protocol.Available +} + +type NearStatus struct { + ChainId string `json:"chain_id"` + Version NearVersion `json:"version"` + SyncInfo NearSyncInfo `json:"sync_info"` +} + +type NearVersion struct { + Version string `json:"version"` +} + +type NearSyncInfo struct { + LatestBlockHeight uint64 `json:"latest_block_height"` + LatestBlockTime string `json:"latest_block_time"` + EarliestBlockHeight uint64 `json:"earliest_block_height"` + Syncing bool `json:"syncing"` +} + +var _ validations.HealthValidator = (*NearSyncingValidator)(nil) diff --git a/internal/upstreams/validations/near_validations/near_peers_validator.go b/internal/upstreams/validations/near_validations/near_peers_validator.go new file mode 100644 index 00000000..216727b4 --- /dev/null +++ b/internal/upstreams/validations/near_validations/near_peers_validator.go @@ -0,0 +1,76 @@ +package near_validations + +import ( + "context" + "errors" + "time" + + "github.com/bytedance/sonic" + "github.com/drpcorg/nodecore/internal/protocol" + "github.com/drpcorg/nodecore/internal/upstreams/connectors" + "github.com/drpcorg/nodecore/internal/upstreams/validations" + "github.com/drpcorg/nodecore/pkg/chains" + "github.com/rs/zerolog/log" +) + +var errNearNoPeers = errors.New("near node has no active peers") + +// NearPeersValidator checks network connectivity via `network_info`. +type NearPeersValidator struct { + upstreamId string + connector connectors.ApiConnector + chain *chains.ConfiguredChain + internalTimeout time.Duration +} + +func NewNearPeersValidator( + upstreamId string, + connector connectors.ApiConnector, + chain *chains.ConfiguredChain, + internalTimeout time.Duration, +) *NearPeersValidator { + return &NearPeersValidator{ + upstreamId: upstreamId, + connector: connector, + chain: chain, + internalTimeout: internalTimeout, + } +} + +func (n *NearPeersValidator) Validate() protocol.AvailabilityStatus { + peers, err := n.fetchActivePeers() + if err != nil { + log.Error().Err(err).Msgf("failed to fetch near network info for upstream '%s'", n.upstreamId) + return protocol.Unavailable + } + if peers == 0 { + log.Error().Err(errNearNoPeers).Msgf("near upstream '%s' has no active peers", n.upstreamId) + return protocol.Unavailable + } + return protocol.Available +} + +func (n *NearPeersValidator) fetchActivePeers() (uint64, error) { + ctx, cancel := context.WithTimeout(context.Background(), n.internalTimeout) + defer cancel() + + request, err := protocol.NewInternalUpstreamJsonRpcRequest("network_info", []any{}, n.chain.Chain) + if err != nil { + return 0, err + } + response := n.connector.SendRequest(ctx, request) + if response.HasError() { + return 0, response.GetError() + } + var info nearNetworkInfo + if err := sonic.Unmarshal(response.ResponseResult(), &info); err != nil { + return 0, err + } + return info.NumActivePeers, nil +} + +type nearNetworkInfo struct { + NumActivePeers uint64 `json:"num_active_peers"` +} + +var _ validations.HealthValidator = (*NearPeersValidator)(nil) diff --git a/internal/upstreams/validations/near_validations/near_validators_test.go b/internal/upstreams/validations/near_validations/near_validators_test.go new file mode 100644 index 00000000..a6e7b996 --- /dev/null +++ b/internal/upstreams/validations/near_validations/near_validators_test.go @@ -0,0 +1,129 @@ +package near_validations_test + +import ( + "fmt" + "testing" + "time" + + "github.com/drpcorg/nodecore/internal/protocol" + "github.com/drpcorg/nodecore/internal/upstreams/validations" + "github.com/drpcorg/nodecore/internal/upstreams/validations/near_validations" + "github.com/drpcorg/nodecore/pkg/chains" + "github.com/drpcorg/nodecore/pkg/test_utils/mocks" + "github.com/stretchr/testify/assert" + "github.com/stretchr/testify/mock" +) + +func isStatus(r protocol.RequestHolder) bool { return r.Method() == "status" } +func isNetworkInfo(r protocol.RequestHolder) bool { return r.Method() == "network_info" } + +func statusBody(chainId string, latestBlockTime string, syncing bool) []byte { + return []byte(fmt.Sprintf( + `{"chain_id":"%s","version":{"version":"2.13.1"},"sync_info":{"latest_block_height":207365026,"latest_block_time":"%s","earliest_block_height":207157933,"syncing":%t}}`, + chainId, latestBlockTime, syncing, + )) +} + +func freshBlockTime() string { + return time.Now().Add(-time.Second).Format(time.RFC3339Nano) +} + +func TestNearChainValidatorValidOnMatch(t *testing.T) { + conn := mocks.NewConnectorMock() + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isStatus)). + Return(protocol.NewSimpleHttpUpstreamResponse("1", statusBody("mainnet", freshBlockTime(), false), protocol.JsonRpc)) + v := near_validations.NewNearChainValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, validations.Valid, v.Validate()) +} + +func TestNearChainValidatorFatalOnMismatch(t *testing.T) { + conn := mocks.NewConnectorMock() + // a testnet node behind a config that says mainnet + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isStatus)). + Return(protocol.NewSimpleHttpUpstreamResponse("1", statusBody("testnet", freshBlockTime(), false), protocol.JsonRpc)) + v := near_validations.NewNearChainValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, validations.FatalSettingError, v.Validate()) +} + +func TestNearChainValidatorFatalOnEmptyChainId(t *testing.T) { + conn := mocks.NewConnectorMock() + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isStatus)). + Return(protocol.NewSimpleHttpUpstreamResponse("1", statusBody("", freshBlockTime(), false), protocol.JsonRpc)) + v := near_validations.NewNearChainValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, validations.FatalSettingError, v.Validate()) +} + +func TestNearChainValidatorRetriesOnFetchError(t *testing.T) { + conn := mocks.NewConnectorMock() + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isStatus)). + Return(protocol.NewHttpUpstreamResponseWithError(protocol.ResponseErrorWithData(1, "boom", nil))) + v := near_validations.NewNearChainValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, validations.SettingsError, v.Validate()) +} + +func TestNearSyncingValidatorAvailable(t *testing.T) { + conn := mocks.NewConnectorMock() + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isStatus)). + Return(protocol.NewSimpleHttpUpstreamResponse("1", statusBody("mainnet", freshBlockTime(), false), protocol.JsonRpc)) + v := near_validations.NewNearSyncingValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, protocol.Available, v.Validate()) +} + +func TestNearSyncingValidatorSyncingOnSyncFlag(t *testing.T) { + conn := mocks.NewConnectorMock() + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isStatus)). + Return(protocol.NewSimpleHttpUpstreamResponse("1", statusBody("mainnet", freshBlockTime(), true), protocol.JsonRpc)) + v := near_validations.NewNearSyncingValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, protocol.Syncing, v.Validate()) +} + +func TestNearSyncingValidatorSyncingOnStaleHead(t *testing.T) { + conn := mocks.NewConnectorMock() + // near's stale-head threshold is 1s * 40; the head is 5 minutes old + staleTime := time.Now().Add(-5 * time.Minute).Format(time.RFC3339Nano) + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isStatus)). + Return(protocol.NewSimpleHttpUpstreamResponse("1", statusBody("mainnet", staleTime, false), protocol.JsonRpc)) + v := near_validations.NewNearSyncingValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, protocol.Syncing, v.Validate()) +} + +func TestNearSyncingValidatorUnavailableOnError(t *testing.T) { + conn := mocks.NewConnectorMock() + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isStatus)). + Return(protocol.NewHttpUpstreamResponseWithError(protocol.ResponseErrorWithData(1, "boom", nil))) + v := near_validations.NewNearSyncingValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, protocol.Unavailable, v.Validate()) +} + +func TestNearSyncingValidatorDoesNotCallNetworkInfo(t *testing.T) { + conn := mocks.NewConnectorMock() + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isStatus)). + Return(protocol.NewSimpleHttpUpstreamResponse("1", statusBody("mainnet", freshBlockTime(), false), protocol.JsonRpc)) + v := near_validations.NewNearSyncingValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, protocol.Available, v.Validate()) + conn.AssertNotCalled(t, "SendRequest", mock.Anything, mock.MatchedBy(isNetworkInfo)) +} + +func TestNearPeersValidatorUnavailableOnZeroPeers(t *testing.T) { + conn := mocks.NewConnectorMock() + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isNetworkInfo)). + Return(protocol.NewSimpleHttpUpstreamResponse("1", []byte(`{"num_active_peers":0,"peer_max_count":40}`), protocol.JsonRpc)) + v := near_validations.NewNearPeersValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, protocol.Unavailable, v.Validate()) +} + +func TestNearPeersValidatorAvailableWithPeers(t *testing.T) { + conn := mocks.NewConnectorMock() + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isNetworkInfo)). + Return(protocol.NewSimpleHttpUpstreamResponse("1", []byte(`{"num_active_peers":36,"peer_max_count":40}`), protocol.JsonRpc)) + v := near_validations.NewNearPeersValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, protocol.Available, v.Validate()) +} + +func TestNearPeersValidatorUnavailableOnError(t *testing.T) { + conn := mocks.NewConnectorMock() + conn.On("SendRequest", mock.Anything, mock.MatchedBy(isNetworkInfo)). + Return(protocol.NewHttpUpstreamResponseWithError(protocol.ResponseErrorWithData(1, "boom", nil))) + v := near_validations.NewNearPeersValidator("id", conn, chains.GetChain("near"), time.Second) + assert.Equal(t, protocol.Unavailable, v.Validate()) +} diff --git a/pkg/chains/chains.go b/pkg/chains/chains.go index fd486d8b..4e6dc53a 100644 --- a/pkg/chains/chains.go +++ b/pkg/chains/chains.go @@ -357,6 +357,8 @@ func getMethodSpecName(blockchainType BlockchainType, methodSpecName string) str return "aptos" case Bitcoin: return "bitcoin" + case Near: + return "near" } return "" diff --git a/pkg/methods/methods_spec_test.go b/pkg/methods/methods_spec_test.go index 8390dac4..7d619a3a 100644 --- a/pkg/methods/methods_spec_test.go +++ b/pkg/methods/methods_spec_test.go @@ -260,6 +260,21 @@ func TestBitcoinSpecLoads(t *testing.T) { assert.Equal(t, []string{"bc1qxyz"}, params) } +func TestNearSpecLoads(t *testing.T) { + err := specs.NewMethodSpecLoader().Load() + assert.NoError(t, err) + + spec := specs.GetSpecMethod("near", "block") + assert.NotNil(t, spec) + + spec = specs.GetSpecMethod("near", "send_tx") + assert.NotNil(t, spec) + assert.True(t, spec.IsBroadcastDispatch()) + + spec = specs.GetSpecMethod("near", "broadcast_tx_async") + assert.Nil(t, spec) +} + func TestAptosSpecMatchesNestedAccountAndTableRoutes(t *testing.T) { loader := specs.NewMethodSpecLoader() err := loader.Load() diff --git a/pkg/methods/specs/near-json-rpc.json b/pkg/methods/specs/near-json-rpc.json new file mode 100644 index 00000000..aae82e04 --- /dev/null +++ b/pkg/methods/specs/near-json-rpc.json @@ -0,0 +1,113 @@ +{ + "openrpc": "1.0.0", + "info": { + "title": "NEAR JSON-RPC methods", + "version": "1.0.0" + }, + "spec": { + "name": "near-json-rpc", + "api-connectors": ["json-rpc"], + "type": "plain" + }, + "methods": [ + { + "name": "block", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "chunk", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "tx", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "EXPERIMENTAL_tx_status", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "EXPERIMENTAL_receipt", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "query", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "EXPERIMENTAL_changes", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "EXPERIMENTAL_changes_in_block", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "EXPERIMENTAL_protocol_config", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "gas_price", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "validators", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "status", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "network_info", + "params": [], + "settings": { + "cacheable": false + } + }, + { + "name": "send_tx", + "params": [], + "settings": { + "cacheable": false, + "dispatch": "broadcast" + } + } + ] +} diff --git a/pkg/methods/specs/near.json b/pkg/methods/specs/near.json new file mode 100644 index 00000000..f4d012c7 --- /dev/null +++ b/pkg/methods/specs/near.json @@ -0,0 +1,12 @@ +{ + "openrpc": "1.0.0", + "info": { + "title": "NEAR methods", + "version": "1.0.0" + }, + "spec": { + "name": "near", + "type": "bundle" + }, + "spec-imports": ["near-json-rpc"] +} diff --git a/pkg/test_utils/test_helpers.go b/pkg/test_utils/test_helpers.go index dbf1a818..569736e4 100644 --- a/pkg/test_utils/test_helpers.go +++ b/pkg/test_utils/test_helpers.go @@ -19,6 +19,7 @@ import ( "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/beacon_specific" "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/bitcoin_specific" "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/evm_specific" + "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/near_specific" specific "github.com/drpcorg/nodecore/internal/upstreams/chains_specific/solana_specific" "github.com/drpcorg/nodecore/internal/upstreams/connectors" "github.com/drpcorg/nodecore/internal/upstreams/event_processors" @@ -241,6 +242,16 @@ func NewAztecChainSpecific(ctx context.Context, connector connectors.ApiConnecto return aztec_specific.NewAztecChainSpecificObject(ctx, chains.GetChain("aztec-mainnet"), "id", options, connector) } +func NewNearChainSpecific(ctx context.Context, connector connectors.ApiConnector) *near_specific.NearChainSpecificObject { + options := &chains.Options{ + InternalTimeout: 5 * time.Second, + ValidationInterval: 10 * time.Second, + DisableChainValidation: new(false), + DisableHealthValidation: new(false), + } + return near_specific.NewNearChainSpecificObject(ctx, chains.GetChain("near"), "id", connector, time.Second, options) +} + func NewAlgorandChainSpecific(ctx context.Context, connector connectors.ApiConnector) *algorand_specific.AlgorandChainSpecificObject { options := &chains.Options{ InternalTimeout: 5 * time.Second,