catch up#1
Open
Corey-Code wants to merge 1036 commits into
Open
Conversation
* feat(tron): add claimedvotereward to tron chain extra data * feat(tron): improve perfomance of GetAddressChainExtraData by calling the backend concurrently * tests(tron): add stakingpool fixture to tron fakechain and update public tron test * feat(tron): omit stakinginfo/chainextradata for non-existent/non-activated addresses * fix(tron): tests missing "address" field * feat(tron): show stakinginfo on the UI /account page * fix(tron): calculation of available/total voting power * docs(tron): description of the address.chainextra resource fields * feat(tron): add check for malformed vote count * docs(tron): updated stakingInfo field description * feat(tron): add mutex to MockTronHTTPClient to avoid data races for concurrent methods * fix(tron): getting delegated frozen balance for staking account info * feat(tron): add tron tx memo/note and process it safely * refactor(tron): simplify stakedBalance computation in tronBuildStakingInfo big.Int.Add(x, y) stores the sum in the receiver and does not mutate its operands, so the inner new(big.Int).Set(stakedBandwidth) copy was redundant. Drop it and pass stakedBandwidth directly to Add. No behavior change; same result, one fewer big.Int allocation per call. * fix(tron): clamp negative limit to zero in tronAvailableResource Previously, a negative limit returned by the Tron API combined with a smaller used value would yield a non-sensical positive availability (e.g., limit=10, used=-50 -> 60). All other resource fields in tronBuildStakingInfo already clamp negative values to zero, so make tronAvailableResource symmetric and treat any non-positive limit as zero availability. Real Tron nodes shouldn't return negative limits, but this hardens the function against malformed responses. * docs(tron): document TRON_POWER exclusion from stakedBalance In Stake 2.0, only BANDWIDTH and ENERGY freeze types produce frozen TRX; TRON_POWER is a derived voting weight, not a separate stake. The frozenV2 loop in tronBuildStakingInfo therefore intentionally ignores TRON_POWER entries when summing stakedBalance. Add an inline comment explaining the intent so the silent drop is not mistaken for a missing case. * chore(metrics): increase metric buckets for rpc latency * fix(tron): mempool tx not retrieved using HTTP API * fix(tron): set default GasUsed as "0x0" so it is packed to txcache without error * tests(tron): test correct setting of "GasUsed" in transaction * fix(tron): correctly process mempool txs using the gettransactionfrompending endpoint * fix(tron): tron api does not always return "data" in tx logs * fix(tron): allow the addrDesc to be empty for specific tron transactions (voting, etc.) to avoid parse errors * refactor(tron): more readable max capping of values in tron staking info --------- Co-authored-by: pragmaxim <pragmaxim@gmail.com>
* fix: skip ZMQ initialization when message_queue_binding is empty * refactor(btc-zeromq): refactor the zeromq init code --------- Co-authored-by: Jakub Jerabek <xjerab28@stud.fit.vutbr.cz>
Cap descriptor change lists, bound xpub scan width, and trim the xpub cache by entry count.
Keep change-list parsing broad enough to return explicit limit errors and report descriptor regex compilation failures clearly.
Agent-Logs-Url: https://github.com/trezor/blockbook/sessions/16574986-e362-4eac-8ee5-5ecf9293f9aa Co-authored-by: pragmaxim <8983344+pragmaxim@users.noreply.github.com>
Agent-Logs-Url: https://github.com/trezor/blockbook/sessions/8fda94ae-e21d-46ff-abe7-9bbfa806c56f Co-authored-by: pragmaxim <8983344+pragmaxim@users.noreply.github.com>
* feat: added avalanche priority fees and corrected existing ones * EIP-1559 : fall back to on-chain estimation when the alternative provider returns empty fees * fix(infura): align infura fees period for avax with other evm chains --------- Co-authored-by: pragmaxim <pragmaxim@gmail.com> Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
…ng to ResyncIndex
wsConnection already multiplexes concurrent requests by ID and tests within a coin run sequentially, so the 3-connection lease pool bought nothing over one persistent socket while adding an unbounded acquire retry loop and a permanently-cached failed init. One shared connection with redial-on-close keeps the handshake-per-coin win; every wait is bounded by the dial and message timeouts, and a failed dial is not cached so the next call retries. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… and panel debug_traceBlockByHash is typically sub-100ms, reaches seconds only when the node is overloaded, and ~10-15s observations are trace_timeout / rpc_timeout expiries — not routine latency. Say so in the histogram help text, and exclude failed calls from the Grafana p50/p95 queries so timeout expiries do not skew the quantiles (failures are already charted in the sync RPC errors panel). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…y deletion The generated Debian package installs a daily cron wrapper that runs bin/logrotate.sh as root, while postinst chowns the per-coin log directory to the unprivileged Blockbook service user. The cleanup script expanded the discovered log path unquoted in both `fuser` and `rm -f $log`, so a file whose name contains shell word-splitting tokens (e.g. "blockbook-bitcoin.log -rf /opt/coins/data/bitcoin/backend") would be split into multiple operands. On GNU rm, `-rf` is honored even after an earlier operand, letting the service user cause the root cron job to recursively delete an attacker-chosen path outside the log tree (backend/RocksDB data, config, other coins' data). Quote "$LOGS" and "$log" and pass `--` before operands so an attacker-controlled path is always treated as a single non-option argument. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
deploy is a fail-fast:false matrix, but wait-for-sync and e2e-tests were gated on needs.deploy.result == 'success'. A single failed coin therefore skipped sync verification and e2e coverage for every coin that DID deploy, leaving freshly deployed instances unverified. Run both jobs on partial failure too; only mode=build (skipped) and cancellation keep them off. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Seven *_test.py files (~1000 lines covering runner.py, build/deploy plans, backend policy and wait_for_sync) existed but were executed by no workflow or Makefile target, so regressions in the pipeline scripts shipped with green CI. Run them via unittest discover in the lint job; they are stdlib-only, need no secrets, and are safe for fork PRs on the GitHub-hosted runner. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Lower SYNC_TIMEOUT_SECONDS to 300 and the job timeout-minutes to 6, keeping the one-minute buffer over the script deadline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Only wait-for-sync had a timeout; every other job fell back to GitHub's 360-minute default, so a hung apt lock, unresponsive backend RPC or stalled install could occupy a self-hosted runner for six hours and block everything queued behind it. Ceilings are sized well above normal runtimes so they only trip on genuine hangs. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Pass matrix.coin to deploy-bb-and-backend.sh via an env var instead of
expanding ${{ }} directly into the run: script, matching how every other
dynamic value in this workflow is handled. Set persist-credentials: false
on all checkouts so the GITHUB_TOKEN is not left in .git/config on the
persistent self-hosted runners when post-step cleanup is skipped
(cancelled job, crashed runner agent) — testing.yml already does this on
the same runner pool. Nothing in the build/deploy path performs
authenticated git operations after checkout.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Without a concurrency group, every push to a PR queued a full unit->connectivity->integration chain on the limited self-hosted pool even when a newer commit had already superseded it. Cancel-in-progress applies only to pull_request events; push runs on master/develop always run to completion. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
git ls-remote arguments are tail-matching glob patterns, so 'mas*' passed validation, and 'master' validated even if only 'feature/master' existed -- after which actions/checkout used the literal (nonexistent or different) ref. Check the returned ref name for an exact match and pass '--' so a ref can never be parsed as a git option. Git forbids glob characters in ref names, so no legitimate ref is rejected. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
exc.read() in the HTTPError handlers was unprotected: a connection reset while reading an error body (e.g. proxy 502) raised out of the handler and killed a wait that still had minutes of budget, failing the deploy on one flaky poll. Degrade an unreadable error body to empty instead. Also stop truncating the remaining budget to whole seconds, which declared a timeout with up to a second (and one final poll) left. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The RESOLVED_BRANCH_OR_TAG ternary was copy-pasted as job-level env into all six jobs; a one-sided edit would make jobs resolve different refs. Hoist it to workflow-level env (inputs/github contexts are available there) so it exists once. Also drop the prepare_* coins_csv job outputs, which no downstream job consumes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
A dispatched run only showed "Prepare Build Plan -> Build (production_builder)", so the run gave no hint which coins it actually built or deployed. Surface it in three places: a run-name with mode/coins/env/ref so the runs list says it, the coin list in the Build job name, and a runner->coins markdown table written by build_plan.py / deploy_plan.py to GITHUB_STEP_SUMMARY so it renders on the run's summary page. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pull-path EIP-1559 estimate decoded baseFeePerGas as uint64 and then cast it to int64 for big.NewInt. A value above math.MaxInt64 (~9.22e18 wei) would wrap to a negative big.Int. Unreachable on mainnet today but possible on high-fee L2s. Use new(big.Int).SetUint64() instead, matching the header path in attachBlockGas. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
a2ec3b3 remapped 1inch's four fee tiers onto low/medium/high (discarding the low tier and folding instant into high), so 1inch no longer surfaces a separate instant tier. The panel description still claimed 1inch was a "provider that return[s] four tiers", which is no longer true and would send an on-call engineer chasing a phantom instant-tier discrepancy. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
#1583) * fix(tron): the receipt for unsolidified block should not be set to nil so all token transfers are parsed and indexed correctly * feat(tron): increase number of connections in the HTTP client to speed up syncing * feat(tron): remove zeromq nil check to reduce number of requests per block to speed up syncing * feat(tron): use HTTP endpoint for getting block has by number to fasten sequential sync path * fix(tron): drain HTTP response body before close to enable connection reuse net/http only returns a connection to the idle pool when the response body is read to EOF and then closed. Request() closed the body without draining: the >=300 early return reads nothing, and json.Decoder can leave trailing bytes unconsumed on larger responses, so connections were torn down instead of pooled — defeating the MaxIdleConns/MaxIdleConnsPerHost tuning this branch added to speed up syncing. Drain the body in the deferred close. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * refactor(tron): drop dead isSolidified param from requestBlockHashByNum requestBlockHashByNum is only called from GetBlockHash's fast path, which is guarded by isBlockSolidified() && solidityNodeHTTP \!= nil, so the sole caller always passed true and the /wallet/getblock (full-node) branch was unreachable. Hardcode the solidity-node client and /walletsolidity/getblock path so the function's actual single-purpose role is explicit and no untested branch lingers. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * docs(tron): add initial sync tuning --------- Co-authored-by: pragmaxim <pragmaxim@gmail.com> Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
PIVX Sapling transactions (version >= 3) include shielded fields (valueBalance, vShieldedSpend, vShieldedOutput, bindingSig) after the transparent part. wire.MsgTx.TxHash() only hashes the transparent fields, producing a phantom txid that is stored in the address index. Fix by computing the txid as sha256d of the full raw transaction bytes for Sapling transactions, matching PIVX's consensus definition. Also fix a pre-existing encoding bug in ParseTx: it used WitnessEncoding via Deserialize(), which can misinterpret a Sapling tx with 0 inputs as a segwit transaction (0x00 0x01 marker). Now reads the version first and uses BaseEncoding for version >= 3, matching PivxDecodeTransactions. Fixes #1522
…ndex EthereumRPC.GetTransaction relied solely on eth_getTransactionByHash, which returns null for mined transactions older than the backend's transaction-by-hash retention window on some archive nodes (observed on QuikNode Base, now the Base backend). Such transactions are still fully retained and reachable via eth_getBlockByHash and eth_getTransactionReceipt, but blockbook reported them as ErrTxNotFound, failing both production lookups and the base=main/rpc/GetTransaction integration test. Fall back to reconstructing the transaction from its receipt (for the block hash) plus the full block body when eth_getTransactionByHash returns null. Genuinely unknown transactions still return ErrTxNotFound, and the primary path is unchanged for backends that retain the full index. Fixes #1609 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
recoverMinedTransaction reconstructed a pruned-index transaction by fetching the entire block body (eth_getBlockByHash, fullTxs=true) and scanning it for the hash. On endpoints that call GetTransaction per tx (address/xpub/block/balancehistory), a cold-cache page of transactions from the pruned window fetched and re-parsed the full block body once per tx - the same ~592 KB block up to 1000x on a block page. Fetch the transaction directly by its receipt's (blockHash, transactionIndex) via eth_getTransactionByBlockHashAndIndex - an O(1), ~900x smaller lookup - and reuse the already-fetched receipt for EthTxToTx instead of fetching it a second time. Verified against the QuikNode Base archive backend: the by-index result is byte-identical to the block-body scan across transactions 64-926 days old. Fall back to the block-body scan when the positional lookup is unavailable, so the change is strictly non-regressive for any backend, and give each recovery RPC its own timeout budget. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Add <network>_ALLOWED_EVM_CALL_METHODS, a comma-separated list of 4-byte method selectors that permits websocket rpcCall requests to any address when the calldata starts with an allowed selector (e.g. 0xdd62ed3e for ERC-20 allowance). Complements ALLOWED_RPC_CALL_TO: a call passes when either its target address or its selector is allowed; with neither set rpcCall stays unrestricted. Malformed calldata never matches (full hex decode, fail closed) and malformed or empty selector config fails startup. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Turn ALLOWED_RPC_CALL_TO and ALLOWED_EVM_CALL_METHODS into runtime settings: the environment variables remain the startup defaults, while an override written through the authenticated internal API GET/POST/DELETE /admin/runtime-settings/<KEY> is persisted in RocksDB (own cfDefault rows, independent of the periodically stored internalState blob and invisible to older versions), survives restarts, takes precedence over the environment and takes effect on websocket rpcCall immediately, without a restart. The live allowlists are an immutable snapshot behind an atomic pointer on InternalState, so the rpcCall hot path stays lock-free and both servers (public websocket, internal admin) share one view regardless of which of them a deployment runs. Writes are strictly ordered: validate (400 on invalid input) -> store to DB (500 and untouched live state on failure) -> publish snapshot -> log the change with old/new value, source and client address. POST of an exactly empty value explicitly unconfigures a dimension; whitespace- or separator-only values are rejected so a botched input cannot silently un-restrict rpcCall; DELETE reverts to the environment default and validates it first so a malformed environment cannot brick the next restart. The TO-list parser is unified with the selector parser: entries are trimmed, empty entries skipped and a set-but-empty value fails startup (previously "," allowlisted the empty string, which matched rpcCall requests with an empty to field). A minimal /admin/runtime-settings page lists the current values and their sources with curl hints. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Avoids allocating a decoded copy of the full calldata (up to 4 MiB) on every rpcCall allowlist check. The full hex encoding is still validated byte-by-byte to preserve the existing fail-closed behavior for malformed calldata.
Addresses the API unification review on PR #1597: - POST /admin/contract-info/ returns a typed {"updated":N} object instead of a pre-serialized string that jsonHandler double-encoded on the wire; a malformed body is now a 400 (public APIError) rather than a 500 - apiContractInfo dispatches on method like apiRuntimeSetting: POST/PUT write the collection path only (an address segment is rejected instead of silently ignored), unknown methods are a 400 instead of falling into the GET path - new DELETE /admin/contract-info/<address> purges cached contract metadata (DB row + LRU entry) so it is re-fetched from the backend on the next read; idempotent, missing row reports deleted:false - shared urlPathSegment helper unifies path-segment extraction; contract addresses get no case normalization (the chain parser is the authority), runtime-setting keys keep their uppercasing at the call site - runtime-settings routes and initRpcCallAllowlists move out of the ChainEthereumType guard (the mechanism is chain-generic and the init already runs unconditionally in NewWebsocketServer); contract-info and internal-data-errors stay EVM-gated, nav links follow the same split Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Smaller items from the PR #1597 review: - lowercase the "invalid EVM call method selector" parse error to match the sibling parser's message convention - trim environment values wherever they are resolved (startup and the DELETE fallback), so stray whitespace from an env file never surfaces in values; a set but whitespace-only env value is a configuration error rather than unset — treating it as unset would silently un-restrict rpcCall - document the differing To/Methods map key formats on RpcCallAllowlists so a future allowlist dimension picks a convention deliberately - runtimeSettingStore gains GetRuntimeSetting; resolveRuntimeSetting reads through the interface instead of requiring *db.RocksDB Admin runtime-settings page gets an inline editing UI (edit/save/delete per setting, replacing the curl-only instructions) and the page/nav/ routes are consistently available on all chain types. Hardening found by review of the new DELETE /admin/contract-info/: - bump protocolGen before the cache purge in DeleteContractInfoForAddress so a concurrent GetContractInfo cannot re-insert the deleted row into the LRU (same idiom as SetErcProtocol); without it the purge could be silently undone until eviction or restart - DELETE returns (and logs) the purged record: the backend re-fetch restores only name/symbol/decimals, not the sync-owned createdInBlock/destructedInBlock, so the response gives the operator a POST-back restore path and the docs now state exactly what is discarded - a failed save in the admin UI keeps the row in edit state instead of stranding an orphaned input, and error responses are unwrapped to their message Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Supports the bbctl counterpart (trezor/bbctl#10) of the runtime-settings admin API in a two-replica deployment where each replica owns its DB: - GET /admin/runtime-settings/ (bare collection path) returns every setting with its effective value and source as a JSON array, so a management tool reads the whole state per replica in one request; other methods on the bare path stay a 400 - initRpcCallAllowlists warns at startup when a stored override shadows a different (or malformed) environment value, making env/DB drift — a replica that missed an admin update, or an env change rolled out while an override exists — visible; the shadowed value is only compared, never parsed, so drift cannot fail a restart - docs/env.md explains the deployment roles of the two sources: the env var is the deploy-managed baseline that survives a replica resync (the DB is wiped, and unset allowlists would silently un-restrict rpcCall), the stored override is the runtime layer that survives restarts until the next deploy ships an updated environment Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
GET /admin/contract-info/ (bare collection path) lists the stored
records, mirroring the runtime-settings list convention. Unlike runtime
settings the collection is unbounded — sync stores a record per contract
creation, millions on a busy chain — so the list is paginated:
?limit=<1..10000> (default 1000) with a from cursor, and the response
{"contracts":[...],"next":"<address>"} carries the from of the next
page. ListContractInfos iterates cfContracts in address-descriptor
order (the CF holds only plain addrDesc keys; protocol rows live in
their own CF).
The former 400 for GET of the bare path moves to the list; DELETE and
POST semantics on the bare path are unchanged.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Post-deploy step in the e2e-tests job exercising the admin API (/admin/runtime-settings/, /admin/contract-info/) of the deployed base_archive dev instance, leaving its database and live rpcCall policy exactly as found — at the end of a run and at every instant during it: - runtime settings use a zero-policy-impact roundtrip: the POSTed value is always the current effective value (or the explicit-empty override when unset), so the parsed allowlist never changes; the final DELETE restores the original source; db-sourced settings are only rewritten in place, since their env fallback is unknown from outside - the contract roundtrip uses a reserved never-real address, pre-cleans it (healing a crashed run) and deletes it at the end; the cleanup and restore flags are raised before the POSTs, so a response lost after a server-side commit still gets cleaned up - negative tests: 401 without credentials (503 = admin disabled fails loudly), unknown key, invalid values proven to change nothing, POST to an address path, unsupported method, list pagination limits - target URL derives from BB_DEV_API_URL_HTTP_<coin> plus the coin config's ports.blockbook_internal (BB_ADMIN_E2E_URL overrides for local runs); the coin gate compares against coins_csv deploy aliases (test_coins_csv carries test names and would never match); skips are annotated (::warning:: for a missing BB_RUNTIME_ENV, ::notice:: for coin-not-deployed) so a dead signal cannot look like a pass - credentials are parsed from BB_ENV inside the script (surrounding quotes stripped like systemd EnvironmentFile) and never printed; transient network errors are retried once, then fail with a clean pipeline error; helpers reuse runner.py and wait_for_sync.py Verified live against blockbook-dev3: two consecutive runs green via the derived URL, instance state identical before and after. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The BB_DEV_API_URL_HTTP_<coin> repo variables follow the tests.json test-name convention (BB_DEV_API_URL_HTTP_base for base_archive), so resolving them by the deploy alias failed with "missing BB_DEV_API_URL_HTTP_base_archive". Map the deploy alias through runner.load_test_coin_name (the same helper deploy_plan.py uses) before the lookup; the coin gate keeps comparing deploy aliases. Unit test pins the mapping against the real base_archive.json. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The paginated contract listing discarded the error from GetAddressesFromAddrDesc and fell through to return next="" for a boundary row whose key did not decode to an address. That silent truncation is indistinguishable from a completed listing and drops the remaining rows; in-page rows with no decoded address were likewise returned with an empty Contract. A corrupt key now returns an error, matching the unpackContractInfo error handling in the same loop. Unreachable for well-formed EVM/Tron contract descriptors, but a visible failure beats a confusing partial page. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.