Skip to main content
All articles

Does Switching RPC Nodes Resolve a Failed Transaction?

· 11 min read
Bankware Global Engineering

You send a transaction, but no RPC response arrives. If another node is available, can you simply switch the connection to it?

The other node can handle the next request. But switching nodes does not erase the transaction you already sent. Changing the connection target and checking the transaction's execution result concern different kinds of state.

In transfer request timeouts and retries, we examined what happens when the same business operation runs again. This time, we move down to the connection layer. Let's compare how BXB selects nodes, rechecks nodes marked as unhealthy, and retries a query through another node in its mirror module.

1. What to ask N2 after losing N1's response​

Suppose RPC nodes N1 and N2 connect to the same EVM network. RPC, or remote procedure call, lets a program request work from another system and receive a response. On a blockchain, it is used to submit transactions and query blocks, balances, and transaction results.

The balances of token T are 100 for A and 0 for B. The goal is to send 10 tokens from A to B. T has zero decimal places and no token transfer fee. A separately holds the native coin needed for gas. Let H be the hash of the signed transaction. In this example, assume we have retained the signed data and H for follow-up checks. N1, N2, and H are names for illustration, not records of an actual incident.

We send transaction H to N1, but a communication error prevents us from receiving the submission response. At this point, we do not know whether the connection broke before N1 relayed the transaction or whether N1 had already relayed it to the network and only the response was lost.

An important request we can make to N2 is to query the result of the same H. N2 being available is not a reason to start by creating a new transaction from A to B. If we later confirm that H executed successfully and the tokens moved, we can close this example with the single transfer originally intended.

The flow below outlines the steps. "Awaiting confirmation" is an explanatory label, not a status name from a specific API.

This is an example of a procedure for confirming an outcome. It does not mean that every failed BXB request automatically performs this entire procedure or that BXB stores the hash of every transaction whose response is missing.

2. The node pool selects a target for the next request​

The BXB adapter manages a list of node clients for each network. We will call this list a node pool. The pool is created from nodes enabled in the configuration. When request-handling code asks for a client, the pool selects one.

If all clients are marked healthy, selection cycles through the list in order. This is round-robin selection. If some are marked unhealthy, selection cycles through only the healthy subset. If an existing pool has no healthy candidates, that selection request ends with 503 Service Unavailable.

Current pool stateNext selection
N1 and N2 are both healthySelect the two candidates in turn
N1 is unhealthy, N2 is healthySelect N2
N1 and N2 are both unhealthyReturn an error because no node is available

Healthy and unhealthy in this table are states recorded by the clients. They do not guarantee that the actual node will respond at the moment it is selected or that it holds the latest block.

In the EVM transmission path used in this example, a communication IOException marks the client unhealthy. When other code next asks the pool for a client, the pool can exclude N1 and select N2. But the pool does not take the contents of the call that just failed and execute it through N2 instead. We need to distinguish the point at which a new target is selected from a call already in progress against a selected target.

BXB's transaction result collector also has a path that selects one client for each batch of work on a network and uses that same client to query multiple transactions within the batch. "A different node can be used at the next selection" therefore does not mean "every RPC call immediately moves to another node."

3. What does a healthy status actually check?​

A health check means different things depending on the question it asks. Whether an address is reachable, whether a particular RPC responds, whether the network is the expected one, and whether the node has caught up to a desired block are separate questions.

The BXB EVM health check we reviewed calls eth_chainId and checks for an RPC error or an exception. This RPC returns a chain identifier. However, this health-check method does not compare the returned value with the registered chain identifier, nor does it check the latest block height or synchronization lag. Passing this check should be interpreted within that scope.

Checks also differ across chains. The XRPL path uses server_info to check the expected network when the RPC object is first retrieved, or when it is retrieved again after the previous check's validity period has expired. Cardano has a check method that queries the latest block, while Solana has one that uses its health RPC. The existence of a check method is separate from whether every ordinary request failure automatically causes a node to be excluded.

The common pool does not run a health check against every node before creating its clients. The recovery scheduler also periodically checks only clients already marked unhealthy. Its scope differs from a monitoring loop that continually checks all healthy candidates.

By default, this scheduler runs every minute. The setting can be changed or disabled. It requests checks of unhealthy candidates in parallel. When a check changes a client's state to healthy, that client can be included in subsequent selections again. The default interval is not a promise to detect and recover from any failure within one minute. The path that records a failure and the time spent checking and waiting for responses also affect recovery.

4. Replacing the node list does not move calls already in progress​

An operator may change node settings and rebuild the pool while the system is running. Registering a new list does not automatically give a new client to work that was using an old one. Some work may continue using the objects and connections it already obtained.

When BXB replaces a pool, it does not immediately close the old pool's clients. It schedules them to close 30 seconds later. This grace period avoids immediately cutting off connections that work in progress may still be using.

This differs from waiting until every operation has finished before closing the clients. It does not guarantee completion for work that takes more than 30 seconds, nor does it implement a handoff of that work to another node. When node settings change, three things still need to be considered separately:

Which list new requests use, which connection an existing call continues to use, and which identifier tracks a transaction already submitted. Closing a connection resource does not cancel H after it has been sent to the chain.

5. The mirror tries other nodes within a single query call​

BXB's mirror reads blockchain data and applies it to a separate database. The block and transaction receipt queries discussed in moving chain events into a database are part of this work. Node switching in this module is implemented differently from the adapter pool described above. We should not assume that the two modules share one list of healthy and unhealthy states.

For each call, the mirror's query client starts with its designated starting node. If it encounters certain HTTP errors or client exceptions, it builds a request for the same query purpose against the next candidate. Because each node can use its own configured RPC method, this does not mean the transmitted bytes are always identical. A single call traverses the candidate list at most once. If it cannot obtain a response, it returns an exception.

The branches in the code we reviewed are as follows. This is the policy of this mirror implementation, not a rule that applies to every RPC service.

Observed failureHandling within this mirror call
HTTP 400 or 404Return an error without trying the next candidate
Other HTTP 4xx or 5xxTry the next candidate
RestClientExceptionTry the next candidate

HTTP 429 is included in the second row. The final row covers a family of HTTP client exceptions, including connection errors. Even with a fixed number of nodes, the waiting time for each request is a separate matter. A bound on attempts does not guarantee a bound on total processing time.

For example, while querying the receipt for the same H, N1 may return HTTP 503 and N2 may return a normal response. That query call can then return the response received from N2. What changed was the route used to read an existing result. No new transaction was created to send tokens.

The mirror also keeps the starting index for the next call in memory. When another candidate succeeds, it updates the starting point toward the position after that successful candidate. It would therefore be inaccurate to say that all later requests stay pinned to the first node that succeeds. This is also a different mechanism from the adapter pool's recurring checks, exclusion, and recovery.

6. An HTTP response may not contain the result you need​

Receiving an HTTP response from another node does not immediately mean we have the required data. JSON-RPC places either a successful result or an error in the response body. The JSON-RPC specification distinguishes the roles of these two fields. We need to check separately whether the HTTP exchange completed and whether the requested RPC succeeded.

Even a successful response can contain an empty result. If H's receipt has not yet been found, the essential part of the response may look like this:

{
"jsonrpc": "2.0",
"id": 1,
"result": null
}

eth_getTransactionReceipt returns null if it cannot find a receipt. A transaction not yet included in a block has no receipt. Receiving null from N2 is not evidence that H will never execute.

The mirror's node traversal loop we reviewed does not decide to switch nodes based on the meaning of a JSON-RPC error or result: null. Some paths check for a missing result in the producer or consumer after receiving the response. But those errors cannot be described as triggering the same automatic retry through HTTP candidates discussed above.

The ledger state each node observes also matters. Even when two nodes on the same network respond, they may differ in the latest block or transaction propagation state they have observed. Receiving blocks at the same height alone does not establish that their contents are identical. Before using another node's response to continue a business process, we need to check the network, transaction hash, source block, and required level of finality.

The current node selection and HTTP traversal do not perform all these checks on our behalf. They also do not resolve missed database updates, duplicates, or chain reorganizations in the mirror. Those remain data-processing responsibilities after connectivity has been restored.

7. Change the connection, preserve the transaction's identity​

Return to H from the opening example. In this illustrative case, assume N1 relayed H to the network and only the response was lost. The first query through N2 found no receipt, but a later query confirms that H executed successfully and 10 units of T moved. This follow-up query assumes a separate confirmation procedure. It does not mean that the mirror's handling of a missing result automatically leads to another query. Once the required finality criteria are also met, the business operation can be marked complete.

If there are no other transactions, the final balances are A 90 / B 10. Gas is paid separately from A's native coin balance. N2 was used to confirm H's result, so switching nodes does not cause another 10 tokens to move.

If H's result is still unknown, its outcome remains under investigation. Even if a query fails because no healthy node is available, that is an error indicating there is no connection available to check the result. It does not establish that H failed on the chain.

The node pool chooses a request target, while the mirror's candidate traversal finds another route for a query. The business system must keep track of the same transaction and the same processing purpose even when that route changes. Keeping these responsibilities distinct lets us use node switching without duplicating the transaction.