Total issuance and burn accounting are broken

[bdmason]

Calculations

total_issuance = subnet_alpha_out + subnet_alpha_in + subnet_alpha_in_provided

burn = subnet_alpha_out - SUM(total_hotkey_alpha) (taking only the total hotkey alpha entries where netuid matches the subnet).

Problem

Burning happens on chain in 2 ways:

1. The burn extrinsic: reducing a coldkey's stake without reducing subnet_alpha_out.
2. Incentive distribution: increasing subnet_alpha_out as normal in run_coinbase.rs but not giving the alpha to the keys owned by the subnet.

There was a 3rd way, in the chain's subsidy mechanism, but that was a small amount and was implemented by accident as it should have been recycling instead, so should probably be ignored.

In both scenarios a diff is increased between the accounting storage of owned alpha on a subnet (subnet_alpha_out), and stake owned by accounts (the sum of total_hotkey_alpha where the netuid matches the subnet). This diff is issued but inaccessible alpha, known as burned alpha.

subnet_alpha_out has been updated incorrectly since dtao launch, leading to the chain not knowing what the correct issuance is on subnets, and not being able to derive the correct burn amount on a subnet.

Evidence

Query the storage on block 6874108 (any block will give similar results, this is just one I picked). 6 subnets have greater total_hotkey_alpha than subnet_alpha_out. 6 subnets have a negative burn, and root accounting is off by ~562k TAO.

  {
    "netuid": 0,
    "diff": "-561931452490774"
  },
  {
    "netuid": 17,
    "diff": "-5409469998121"
  },
  {
    "netuid": 32,
    "diff": "-3529669489799"
  },
  {
    "netuid": 55,
    "diff": "-2585186709697"
  },
  {
    "netuid": 53,
    "diff": "-312311125448"
  },
  {
    "netuid": 20,
    "diff": "-216796023194"
  },
  {
    "netuid": 2,
    "diff": "-180977417688"
  },

If you query block 4929350, which is roughly a day after dtao launch, the diff is negative for every single subnet.

Impact

• In the distant future, the alpha halvenings will happen at the wrong time.
• In the even further distant future the max supply of 21m tokens will not be respected.
• Right now nobody knows the correct amount of alpha burned without manually indexing the chain through the various burning mechanisms that have been implemented.

I believe we have the correct burn totals at Taostats, but it's a very complex indexer so there could be errors I haven't spotted.

Taking subnet 17 as an example, taostats (~67k) & taomarketcap (~71k) have a similar burn amounts (less than 5% difference), tao.app shows 0, the chain shows -5409 alpha burned.

[antlerminator]

Indeed stuff ain't close to being correct. Relevant issue: #1956

[bdmason]

Indeed stuff ain't close to being correct. Relevant issue: #1956

Spec version 240 was released on 19th Feb, I saw subnet_alpha_out appearing smaller than total_hotkey_alpha on the day of dtao launch (13th Feb). I'm not sure that issue alone is the problem. I'm not saying that release isn't a problem, just that there could be multiple problems.

[Dairus01]

I just fixed that problem in my pull request here

#2297

The core accounting bug stems from a desynchronization between individual user ledgers and global subnet ledgers during specific "admin-like" operations (Burning and Recycling).

In the Subtensor architecture, two layers of accounting must always move in unison:

1. Individual Stake: How much Alpha a specific (Hotkey, Coldkey) pair owns.
2. Subnet Alpha Out: The total amount of Alpha tokens that exist on a specific Subnet.

The Logic Gap

In recycle_alpha.rs, the functions do_recycle_alpha and do_burn_alpha correctly called:

Self::decrease_stake_for_hotkey_and_coldkey_on_subnet(...)

This helper function successfully removed tokens from the User's Balance.

However, these functions failed to decrement the corresponding SubnetAlphaOut global counter.

The Consequence (The Bug)

Because the user's balance went down but the global subnet counter remained unchanged:

• Phantom Liquidity: The subnet believed it had more Alpha tokens than actually existed in user accounts.
• Issuance Drift: Over time, TotalIssuance would drift away from the actual circulating supply.
• Invariant Failure: Sum(UserStakes) != SubnetAlphaOut.

[bdmason]

I just fixed that problem in my pull request here

#2297

The core accounting bug stems from a desynchronization between individual user ledgers and global subnet ledgers during specific "admin-like" operations (Burning and Recycling).

In the Subtensor architecture, two layers of accounting must always move in unison:

1. Individual Stake: How much Alpha a specific (Hotkey, Coldkey) pair owns.
2. Subnet Alpha Out: The total amount of Alpha tokens that exist on a specific Subnet.

The Logic Gap

In recycle_alpha.rs, the functions do_recycle_alpha and do_burn_alpha correctly called:

Self::decrease_stake_for_hotkey_and_coldkey_on_subnet(...)
This helper function successfully removed tokens from the User's Balance.

However, these functions failed to decrement the corresponding SubnetAlphaOut global counter.

The Consequence (The Bug)

Because the user's balance went down but the global subnet counter remained unchanged:

• Phantom Liquidity: The subnet believed it had more Alpha tokens than actually existed in user accounts.
• Issuance Drift: Over time, TotalIssuance would drift away from the actual circulating supply.
• Invariant Failure: Sum(UserStakes) != SubnetAlphaOut.

Your PR doesn't solve the issue. I've explained here #2297 (review).

[Dairus01]

Check the latest fix now

[bdmason]

Check the latest fix now

it definitely doesn't fix the problem

[Dairus01]

Check the latest fix now

it definitely doesn't fix the problem

So what exactly would fix the problem

[bdmason]

So what exactly would fix the problem

I don't know, it could be multiple things. The issue has been around since Feb 13th, and the run_coinbase.rs file has been through many iterations so it's extremely difficult to pin down.

I don't intend to be disrespect your effort to solve the problem, but I explained 2 days ago that the issue can't be in the recycle alpha method as it has never been called successfully on chain. Since then you persisted with fixes in the same method, and now the latest fix you propose has no effect on the problematic storage item SubnetAlphaOut. It doesn't make sense.