Skip to content

Update getTxInfo endpoint API #204

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
samholmes wants to merge 1 commit into
base: master
Choose a base branch
from
+106 −49
Open

Update getTxInfo endpoint API #204

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 22 additions & 1 deletion src/initDbs.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,14 @@
import {
DatabaseSetup,
JsDesignDocument,
makeJsDesign,
makeMangoIndex,
MangoDesignDocument,
setupDatabase
} from 'edge-server-tools'

import { config } from './config'
import { fixJs } from './util/fixJs'

interface DesignDocumentMap {
[designDocName: string]: MangoDesignDocument | JsDesignDocument
Expand Down Expand Up @@ -73,7 +75,26 @@ const transactionsDatabaseSetup: DatabaseSetup = {
name: 'reports_transactions',
options: { partitioned: true },
documents: {
...transactionIndexes
...transactionIndexes,
'_design/getTxInfo': makeJsDesign(
'payoutHashfixByDate',
({ emit }) => ({
map: function(doc) {
const space = 1099511627776 // 5 bytes of space; 2^40
const prime = 769 // large prime number
let hashfix = 0 // the final hashfix
for (let i = 0; i < doc.payoutAddress.length; i++) {
const byte = doc.payoutAddress.charCodeAt(i)
hashfix = (hashfix * prime + byte) % space
}
emit([hashfix, doc.isoDate], doc._id)
}
}),
{
fixJs,
partitioned: false
}
)
}
}
const progressCacheDatabaseSetup: DatabaseSetup = {
Expand Down
122 changes: 74 additions & 48 deletions src/routes/v1/getTxInfo.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@ import Router from 'express-promise-router'
import { reportsTransactions } from '../../indexApi'
import { asDbTx, DbTx, Status } from '../../types'
import { EdgeTokenId } from '../../util/asEdgeTokenId'
import { asNumberString } from '../../util/asNumberString'
import { HttpError } from '../../util/httpErrors'
import { trial } from '../../util/trail'

Expand All @@ -12,7 +13,7 @@ const asGetTxInfoReq = asObject({
* Prefix of the destination address.
* Minimum 3 character; Maximum 5 characters.
*/
addressPrefix: asString,
addressHashfix: asNumberString,

/**
* ISO date string to start searching for transactions.
Expand All @@ -26,18 +27,24 @@ const asGetTxInfoReq = asObject({
})

interface TxInfo {
providerId: string
orderId: string
isoDate: string
sourceAmount: number // exchangeAmount units
sourceCurrencyCode: string
sourcePluginId?: string
sourceTokenId?: EdgeTokenId
swapInfo: SwapInfo

deposit: AssetInfo
payout: AssetInfo
}

interface AssetInfo {
address: string
pluginId: string
tokenId: EdgeTokenId
amount: number
}

interface SwapInfo {
orderId: string
pluginId: string
status: Status
destinationAddress?: string
destinationAmount: number // exchangeAmount units
destinationPluginId?: string
destinationTokenId?: EdgeTokenId
}

export const getTxInfoRouter = Router()
Expand All @@ -50,62 +57,81 @@ getTxInfoRouter.get('/', async function(req, res) {
}
)

if (query.addressPrefix.length < 3) {
res.status(400).send('addressPrefix must be at least 3 characters')
if (query.addressHashfix < 0 || query.addressHashfix > 2 ** 40) {
res.status(400).send('addressHashfix must be between 0 and 2^40')
return
}

const startDate = new Date(query.startIsoDate)
const endDate = new Date(query.endIsoDate)
const startKey = [query.addressHashfix, startDate.toISOString()]
Copy link
Member

@paullinator paullinator Jul 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

start and end have the same hash. This doesn't allow for any privacy to ensure we don't exactly know the users addresss.

If this were a standard sha256 hash into hex, we could use the first N chars as the prefix to allow for privacy

Copy link
Collaborator Author

@samholmes samholmes Jul 22, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The hashfix is not collision resistant. It is a "hash prefix" because it takes 5 bytes from the address (5 bytes is 5 chars). The collision space is 5 bytes, or 40 bits (2^40). We can lower the collision space to 3-bytes which would have many more collisions.

Copy link
Collaborator Author

@samholmes samholmes Jul 24, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OTS comments:

  1. Use a large enough space size of (I recall 64^6)
  2. Use mango queries to do range queries on the hashfix value

The goal is to be flexible in our design. We want to query different collision space over time rather than be locked to some byte size. To do this we need to do range queries on the hashfix. So the space for the hashfix is a maximum, but the range will determine the actual collision space.

Copy link
Collaborator Author

@samholmes samholmes Jan 14, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sidebar Today:

Because we cannot do multiple range queries on a single view key, we're instead going to store the real hash of transaction's addresses. Then with the full hash of the address we can index it and use a mango query to range query it and the date at the same time.

Next, we want to add a new endpoint that gives use the hash prefix length which MUST be used. If the client doesn't use the same prefix length that the server uses, then it'll reject the query. This is to prevent data miners from getting all our transaction data (DOS protection). This endpoint does some average addresses per minute heuristic and returns the prefix that should be used for the number of minutes within the range that would match the average. This way ever query should be hidden within the average.

Here's what Opus 4.5 generated explaining the limitation:

Good question! Hashfix can be a derived value in a view key — the current implementation already does this. The issue is how CouchDB view queries work with composite keys.

The B-tree Ordering Problem

CouchDB views use B-tree ordering on composite keys. With a key like [hashfix, isoDate]:

[100, "2025-01-01"]
[100, "2025-06-15"]
[100, "2025-12-31"]
[101, "2024-01-01"]  ← hashfix 101 comes AFTER all of hashfix 100
[101, "2025-03-20"]
[102, "2024-06-01"]
...

If you query with:

startKey: [100, "2025-01-01"]
endKey:   [200, "2025-12-31"]

You'd get:

  • ✅ hashfix 100, dates from 2025-01-01 onward
  • ❌ hashfix 101-199, ALL dates (not just 2025)
  • ✅ hashfix 200, dates up to 2025-12-31

The date filter only applies at the boundaries, not across the entire range. This is because B-tree range queries work on the key as a tuple, not as independent fields.

Why Mango is Different

Mango's $and selector applies each constraint independently:

{
  payoutHashfix: { $gte: 100, $lte: 200 },
  isoDate: { $gte: "2025-01-01", $lte: "2025-12-31" }
}

This properly filters: "hashfix in range AND date in range" for every document.

View Workaround?

You could make views work with range hashfix, but it's awkward:

  1. Reverse the key to [isoDate, hashfix] — but then you lose efficient hashfix range queries
  2. Post-filter in application code — fetch all hashfixes in range, then filter by date
  3. Multiple queries — one query per hashfix value (very inefficient for large ranges)

So the derived value itself isn't the problem — it's the composite key ordering that makes independent range queries on multiple fields impractical with views.

const endKey = [query.addressHashfix, endDate.toISOString()]

const addressKey = query.addressPrefix.slice(
0,
Math.min(query.addressPrefix.length, 5)
const results = await reportsTransactions.view(
'getTxInfo',
'payoutHashfixByDate',
{
start_key: startKey,
end_key: endKey,
inclusive_end: true,
include_docs: true
}
)

const results = await reportsTransactions.find({
selector: {
payoutAddress: {
$gte: addressKey,
$lte: addressKey + '\uffff'
},
isoDate: {
$gte: startDate.toISOString(),
$lte: endDate.toISOString()
}
const rows = results.rows
.map(row => asMaybe(asDbTx)(row.doc))
.filter((item): item is DbTx => item != null)

const txs: TxInfo[] = rows.map(row => {
const swapInfo = getSwapInfo(row)

const deposit: AssetInfo = {
// TODO: Remove empty strings after db migration
address: row.depositAddress ?? '',
pluginId: '',
tokenId: '',
amount: row.depositAmount
}
})

const rows = results.docs
.map(doc => asMaybe(asDbTx)(doc))
.filter((item): item is DbTx => item != null)
const payout: AssetInfo = {
// TODO: Remove empty strings after db migration
address: row.payoutAddress ?? '',
pluginId: '',
tokenId: '',
amount: row.payoutAmount
}

const result: TxInfo = {
swapInfo,
deposit,
payout,
isoDate: row.isoDate
}

const txs: TxInfo[] = rows.map(row => ({
providerId: getProviderId(row),
orderId: row.orderId,
isoDate: row.isoDate,
sourceAmount: row.depositAmount,
sourceCurrencyCode: row.depositCurrency,
status: row.status,
// TODO: Infer the pluginId and tokenId from the document:
// sourcePluginId?: string,
// sourceTokenId?: EdgeTokenId,
destinationAddress: row.payoutAddress,
destinationAmount: row.payoutAmount
// TODO: Infer the pluginId and tokenId from the document:
// destinationPluginId?: string
// destinationTokenId?: EdgeTokenId
}))
return result
})

res.send({
txs
})
})

/*
Returns the providerId from the document id.
Returns the pluginId from the document id.
For example: edge_switchain:<orderId> -> switchain
*/
function getProviderId(row: DbTx): string {
function getPluginId(row: DbTx): string {
return row._id?.split(':')[0].split('_')[1] ?? ''
}

function getSwapInfo(row: DbTx): SwapInfo {
const pluginId = getPluginId(row)
const orderId = row.orderId
const status = row.status

return {
orderId,
pluginId,
status
}
}
10 changes: 10 additions & 0 deletions src/util/asNumberString.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
import { asString, Cleaner } from 'cleaners'

export const asNumberString: Cleaner<number> = (v: unknown): number => {
const str = asString(v)
const n = Number(str)
if (n.toString() !== str) {
throw new TypeError('Expected number string')
}
return n
}
Loading