transactions, transactions_accounts: Use TOID instead of hash as reference and avoid storing txn hash multiple times#478
Open
aditya1702 wants to merge 13 commits intostatechanges-remove-jsonbfrom
Open
Conversation
There was a problem hiding this comment.
Pull request overview
This pull request refactors the database schema to use transaction order ID (TOID) as the primary key for transactions instead of hash, reducing redundant storage of transaction hashes in the transactions_accounts junction table. The PR changes the primary identifier for tracking transaction-account relationships from hash-based to ToID-based references throughout the codebase.
Changes:
- Modified database schema to make
to_idthe primary key andhasha unique column in thetransactionstable - Updated
transactions_accountstable to referencetx_to_idinstead oftx_hash - Refactored all in-memory data structures from hash-based maps to ToID-based maps throughout the indexer, services, and GraphQL layers
Reviewed changes
Copilot reviewed 20 out of 20 changed files in this pull request and generated no comments.
Show a summary per file
| File | Description |
|---|---|
| internal/db/migrations/2025-06-10.2-create_indexer_table_transactions.sql | Changed primary key from hash to to_id; updated transactions_accounts to reference tx_to_id |
| internal/db/migrations/2025-06-10.3-create_indexer_table_operations.sql | Added CHECK constraints for operation types; added ON DELETE CASCADE |
| internal/db/migrations/2025-06-10.4-create_indexer_table_state_changes.sql | Added CHECK constraints for state change categories and reasons; added ON DELETE CASCADE |
| internal/data/transactions.go | Updated BatchInsert and BatchCopy to use stellarAddressesByToID map; changed SQL to use tx_to_id |
| internal/data/transactions_test.go | Updated test helper to generate ToIDs using toid.New; changed all test assertions to use ToID |
| internal/data/accounts.go | Renamed BatchGetByTxHashes to BatchGetByToIDs; updated query to use tx_to_id |
| internal/data/accounts_test.go | Updated test to verify BatchGetByToIDs functionality with ToID references |
| internal/data/operations_test.go | Updated test data inserts to use uppercase operation types per new CHECK constraint |
| internal/data/statechanges_test.go | Updated test data inserts to use uppercase categories per new CHECK constraint |
| internal/indexer/indexer_buffer.go | Changed participantsByTxHash field to participantsByToID with int64 keys |
| internal/indexer/indexer_buffer_test.go | Updated all tests to set ToID field and verify ToID-based participant tracking |
| internal/indexer/indexer.go | Updated interface to return map[int64]set.Set[string] for transaction participants |
| internal/indexer/indexer_test.go | Updated to use ToID for participant verification |
| internal/indexer/types/types.go | Renamed AccountWithTxHash to AccountWithToID; changed field from TxHash to ToID |
| internal/services/ingest.go | Changed transaction participant maps from string keys to int64 (ToID) keys |
| internal/services/ingest_test.go | Updated test to access participants by ToID instead of hash |
| internal/serve/graphql/resolvers/transaction.resolvers.go | Updated to use ToID in dataloader key instead of TxHash |
| internal/serve/graphql/dataloaders/loaders.go | Renamed AccountsByTxHashLoader to AccountsByToIDLoader |
| internal/serve/graphql/dataloaders/account_loaders.go | Updated loader to use ToID; renamed function and updated implementation |
| internal/integrationtests/infrastructure/backfill_helpers.go | Updated JOIN conditions to use t.to_id = ta.tx_to_id |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request refactors the database schema to use transaction order ID (TOID) as the primary key for transactions instead of hash, reducing redundant storage of transaction hashes in the transactions_accounts junction table. The PR changes the primary identifier for tracking transaction-account relationships from hash-based to ToID-based references throughout the codebase.
Known limitations
N/A
Issue that this PR addresses
Closes #481