feat: add mempool API endpoints and WebSocket support#235
Conversation
Introduce mempool functionality across HTTP JSON-RPC, REST routes and WebSocket: add new JSON-RPC methods (btc_getMempoolInfo, btc_getPendingTransaction, btc_getLatestPendingTransactions), request/result TypeScript interfaces, and API routes (GetMempoolInfo, GetPendingTransaction, GetLatestPendingTransactions) with parameter parsing, address resolution and configured limits. Update protobuf (OPNetAPIProtocol) with mempool messages and add MEMPOOL subscription type. Add WebSocket opcodes, packet types, handlers, and subscription support plus server-side notification flow (BroadcastTransaction triggers WSManager.onMempoolTransaction; WebSocketManager dispatches NewMempoolTransactionNotification to subscribed clients). Add MempoolTransactionConverter to map storage objects to API shapes and wire routes into DefinedRoutes and HandlerRegistry. Add default API.MEMPOOL config values and begin config loader validation for mempool settings.
Introduce mempool RPC and subscription support across protobuf and TypeScript APIs. Added protobuf messages for mempool queries and notifications (GetMempoolInfoRequest/Response, GetPendingTransactionRequest, MempoolTransactionInput/Output, PendingTransactionResponse, GetLatestPendingTransactionsRequest/Response, SubscribeMempoolRequest/Response, NewMempoolTransactionNotification). Update JSON-RPC and WebSocket enums and packet types to expose mempool methods and responses, plus a new server push opcode for mempool transactions. Also add brief docs/comments and a handler registration stub and WebSocketManager broadcast doc for new mempool notifications.
There was a problem hiding this comment.
Pull request overview
This PR introduces comprehensive mempool functionality to the OPNet node, enabling clients to query mempool state and subscribe to new transaction notifications via HTTP JSON-RPC, REST, and WebSocket interfaces.
Changes:
- Added three mempool API endpoints:
btc_getMempoolInfo(aggregate stats),btc_getPendingTransaction(single tx lookup), andbtc_getLatestPendingTransactions(filtered transaction list with optional address resolution) - Implemented WebSocket support with subscription type MEMPOOL, query handlers, notification broadcasting via
WebSocketManager.onMempoolTransaction, and protobuf message definitions - Added configuration section
API.MEMPOOLwith MAX_ADDRESSES, DEFAULT_LIMIT, and MAX_LIMIT parameters, including validation and default values
Reviewed changes
Copilot reviewed 30 out of 30 changed files in this pull request and generated 11 comments.
Show a summary per file
| File | Description |
|---|---|
| src/src/vm/storage/databases/VMMongoStorage.ts | Implements mempool query methods delegating to MempoolRepository |
| src/src/vm/storage/VMStorage.ts | Adds abstract mempool method signatures |
| src/src/db/repositories/MempoolRepository.ts | Implements getMempoolInfo and getLatestTransactions with MongoDB aggregation |
| src/src/config/interfaces/IBtcIndexerConfig.ts | Defines APIMempoolConfig interface with limits |
| src/src/config/BtcIndexerConfigLoader.ts | Adds default config values and type validation for MEMPOOL settings |
| src/src/api/websocket/types/requests/WebSocketRequestTypes.ts | Defines mempool request interfaces for WebSocket |
| src/src/api/websocket/types/opcodes/WebSocketOpcodes.ts | Adds mempool opcodes and opcode-to-name mappings |
| src/src/api/websocket/types/enums/SubscriptionType.ts | Adds MEMPOOL subscription type enum value |
| src/src/api/websocket/packets/types/APIPacketTypes.ts | Defines mempool packet type enums |
| src/src/api/websocket/handlers/HandlerRegistry.ts | Registers mempool query and subscription handlers |
| src/src/api/websocket/WebSocketManager.ts | Implements onMempoolTransaction notification broadcast |
| src/src/api/websocket/OpcodeRegistry.ts | Registers mempool packet builders and opcodes |
| src/src/api/routes/api/v1/transaction/BroadcastTransaction.ts | Triggers mempool WebSocket notifications on successful broadcast |
| src/src/api/routes/api/v1/mempool/MempoolTransactionConverter.ts | Converts database objects to API response format |
| src/src/api/routes/api/v1/mempool/GetPendingTransaction.ts | Implements single transaction lookup endpoint |
| src/src/api/routes/api/v1/mempool/GetMempoolInfo.ts | Implements mempool statistics endpoint |
| src/src/api/routes/api/v1/mempool/GetLatestPendingTransactions.ts | Implements filtered transaction list with address resolution |
| src/src/api/routes/DefinedRoutes.ts | Registers mempool route instances |
| src/src/api/json-rpc/types/interfaces/results/mempool/*.ts | Defines result type interfaces for mempool methods |
| src/src/api/json-rpc/types/interfaces/params/mempool/*.ts | Defines parameter type interfaces for mempool methods |
| src/src/api/json-rpc/types/enums/JSONRpcMethods.ts | Adds mempool JSON-RPC method names |
| src/src/api/json-rpc/routes/JSONRpcRoute.ts | Maps mempool JSON-RPC methods to routes |
| src/src/api/enums/Routes.ts | Defines mempool route enum values |
| src/protocols/OPNetAPIProtocol.proto | Defines protobuf messages for mempool operations |
| src/config/btc.sample.conf | Documents mempool API configuration section |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
||
| if (data) { | ||
| this.safeJson(res, 200, data); | ||
| } else { | ||
| this.safeJson(res, 400, { error: 'Pending transaction not found.' }); | ||
| } |
There was a problem hiding this comment.
The getData method throws an error when a transaction is not found (line 42), so the condition if (data) on line 77 will always be true. The else block on lines 79-80 is unreachable. Consider removing the unreachable else block or adjusting the error handling to return undefined instead of throwing an error.
| if (data) { | |
| this.safeJson(res, 200, data); | |
| } else { | |
| this.safeJson(res, 400, { error: 'Pending transaction not found.' }); | |
| } | |
| this.safeJson(res, 200, data); |
|
|
||
| // Notify mempool subscribers of the new transaction | ||
| if (mergedResult.success && mergedResult.result) { | ||
| WSManager.onMempoolTransaction(mergedResult.result, true); |
There was a problem hiding this comment.
The WebSocket notification is hardcoded to always send isOPNet=true, but not all transactions that successfully broadcast are OPNet transactions. This could mislead WebSocket subscribers about which transactions are OPNet-specific. The isOPNet flag should be determined from the actual transaction verification result or stored mempool data, not hardcoded.
| WSManager.onMempoolTransaction(mergedResult.result, true); | |
| const isOPNet = | |
| (mergedResult as any).isOPNet ?? | |
| (verification as any).isOPNet ?? | |
| (result as any).isOPNet ?? | |
| false; | |
| WSManager.onMempoolTransaction(mergedResult.result, isOPNet); |
|
|
||
| const hash = req.query.hash as string; | ||
|
|
||
| if (!hash || (hash && hash.length !== 64)) { |
There was a problem hiding this comment.
The validation logic if (!hash || (hash && hash.length !== 64)) is redundant. If hash is falsy, the second condition will not be evaluated. If hash is truthy, the first part is unnecessary. Simplify to if (!hash || hash.length !== 64).
| if (!hash || (hash && hash.length !== 64)) { | |
| if (!hash || hash.length !== 64) { |
| private verifyAPIMempoolConfig( | ||
| parsedConfig: Partial<IBtcIndexerConfig['API']['MEMPOOL']>, | ||
| ): void { | ||
| if ( | ||
| parsedConfig.MAX_ADDRESSES !== undefined && | ||
| typeof parsedConfig.MAX_ADDRESSES !== 'number' | ||
| ) { | ||
| throw new Error(`Oops the property API.MEMPOOL.MAX_ADDRESSES is not a number.`); | ||
| } | ||
|
|
||
| if ( | ||
| parsedConfig.DEFAULT_LIMIT !== undefined && | ||
| typeof parsedConfig.DEFAULT_LIMIT !== 'number' | ||
| ) { | ||
| throw new Error(`Oops the property API.MEMPOOL.DEFAULT_LIMIT is not a number.`); | ||
| } | ||
|
|
||
| if (parsedConfig.MAX_LIMIT !== undefined && typeof parsedConfig.MAX_LIMIT !== 'number') { | ||
| throw new Error(`Oops the property API.MEMPOOL.MAX_LIMIT is not a number.`); | ||
| } | ||
| } |
There was a problem hiding this comment.
The configuration validation only checks if the values are numbers but does not validate that they are positive. Negative or zero values for MAX_ADDRESSES, DEFAULT_LIMIT, or MAX_LIMIT could cause unexpected behavior. Consider adding validation to ensure these values are positive integers (greater than 0).
| let addresses: string[] | undefined = decoded.addresses; | ||
| const limit = Math.max(1, Math.min(decoded.limit, Config.API.MEMPOOL.MAX_LIMIT)); | ||
|
|
||
| // If a single address is provided, auto-resolve all address types | ||
| if (decoded.address && !addresses) { | ||
| addresses = await this.resolveAddresses(decoded.address); | ||
| } |
There was a problem hiding this comment.
When both decoded.address and decoded.addresses are provided, the code only uses decoded.addresses (line 51) and ignores decoded.address because of the condition !addresses on line 55. This behavior should be documented or the API should either merge both parameters or reject requests with both parameters to avoid confusion.
| return { | ||
| count: result.count, | ||
| opnetCount: result.opnetCount, | ||
| size: BigInt(result.size), |
There was a problem hiding this comment.
The handler converts result.size to BigInt on line 478, but the protobuf definition (GetMempoolInfoResponse) expects uint64 for the size field (line 666 in OPNetAPIProtocol.proto), and the MempoolRepository returns a plain number. This conversion is correct for protobuf compatibility with large numbers, but ensure the protobuf serializer handles BigInt properly.
| size: BigInt(result.size), | |
| size: result.size, |
| async (request: PackedMessage<GetLatestPendingTransactionsWsRequest>) => { | ||
| const route = DefinedRoutes[Routes.MEMPOOL_TRANSACTIONS] as GetLatestPendingTransactions; | ||
| const result = await route.getData({ | ||
| address: request.address || undefined, |
There was a problem hiding this comment.
The expression request.address || undefined on line 500 will convert empty strings to undefined, which may not be the intended behavior. If an empty string should be treated as invalid input, it would be better to explicitly check for it. Consider using request.address ? request.address : undefined or adding validation to reject empty strings.
| address: request.address || undefined, | |
| address: request.address ?? undefined, |
| if (data) { | ||
| this.safeJson(res, 200, data); | ||
| } else { | ||
| this.safeJson(res, 400, { error: 'Could not fetch pending transactions.' }); | ||
| } |
There was a problem hiding this comment.
The getData method always returns a result (line 66) and never returns undefined, making the condition if (data) on line 101 always true. The else block on lines 103-104 is unreachable. Consider removing the unreachable else block or adjusting the error handling.
| if (data) { | |
| this.safeJson(res, 200, data); | |
| } else { | |
| this.safeJson(res, 400, { error: 'Could not fetch pending transactions.' }); | |
| } | |
| this.safeJson(res, 200, data); |
| if (data) { | ||
| this.safeJson(res, 200, data); | ||
| } else { | ||
| this.safeJson(res, 400, { error: 'Could not fetch mempool info.' }); | ||
| } |
There was a problem hiding this comment.
The getData method always returns a result (line 36-40) and never returns undefined, making the condition if (data) on line 66 always true. The else block on lines 68-69 is unreachable. Consider removing the unreachable else block.
| if (data) { | |
| this.safeJson(res, 200, data); | |
| } else { | |
| this.safeJson(res, 400, { error: 'Could not fetch mempool info.' }); | |
| } | |
| this.safeJson(res, 200, data); |
| const result = await route.getData({ | ||
| address: request.address || undefined, | ||
| addresses: request.addresses?.length ? request.addresses : undefined, | ||
| limit: request.limit || undefined, |
There was a problem hiding this comment.
The expression request.limit || undefined on line 502 will convert 0 to undefined, which may not be the intended behavior. If a user explicitly passes 0 as a limit, it gets replaced with the default limit instead of being treated as an error or returning no results. Consider explicit null/undefined checks instead of relying on falsy coercion.
| limit: request.limit || undefined, | |
| limit: request.limit ?? undefined, |
Description
Introduce mempool functionality across HTTP JSON-RPC, REST routes and WebSocket: add new JSON-RPC methods (btc_getMempoolInfo, btc_getPendingTransaction, btc_getLatestPendingTransactions), request/result TypeScript interfaces, and API routes (GetMempoolInfo, GetPendingTransaction, GetLatestPendingTransactions) with parameter parsing, address resolution and configured limits. Update protobuf (OPNetAPIProtocol) with mempool messages and add MEMPOOL subscription type. Add WebSocket opcodes, packet types, handlers, and subscription support plus server-side notification flow (BroadcastTransaction triggers WSManager.onMempoolTransaction; WebSocketManager dispatches NewMempoolTransactionNotification to subscribed clients). Add MempoolTransactionConverter to map storage objects to API shapes and wire routes into DefinedRoutes and HandlerRegistry. Add default API.MEMPOOL config values and begin config loader validation for mempool settings.
Type of Change
Checklist
Build & Tests
npm installcompletes without errorsnpm run buildcompletes without errorsnpm testpasses all testsCode Quality
Documentation
Security
OPNet Node Specific
Testing
Consensus Impact
Related Issues
By submitting this PR, I confirm that my contribution is made under the terms of the project's license.