Skip to content

Conversation

@Dairus01
Copy link

Subtensor Invariant Enforcement: Problem, Analysis & Fix

1. The Problem: Silent Economic Corruption

In a complex Delegated Proof of Stake (DPoS) system like Subtensor, accounting bugs can be catastrophic. If the total stake stored in SubnetAlphaOut drifts from the actual sum of individual neuron stakes (TotalHotkeyAlpha), or if emission injection exceeds the protocol's defined schedule (BlockEmission), the network suffers from silent economic corruption.

Prior to this fix, such inconsistencies could persist undetected, potentially allowing malicious actors to exploit infinite money glitches or causing catastrophic inflation.

2. Thought Process & Design Decisions

Q: Where should we check for consistency?

Decision: on_finalize hook.
Rationale: Invariants must hold true at the end of every block after all extrinsics and internal logic (like run_coinbase) have executed. Checking at on_initialize is too late (corruption happened in previous block), and checking inside every extrinsic is too heavyweight.

Q: How do we balance safety vs. liveness?

Decision: Pause, Don't Panic (in Production).
Rationale:

  • Panic: If we panic in on_finalize, the block is discarded. If this happens deterministically, the chain halts. This is acceptable for testnets but fatal for mainnet.
  • event + Pause: In production, we detect the violation, emit a Critical event, and pause emissions for the affected subnet. This "stops the bleeding" without stopping the blockchain.
  • Panic (Test): In cfg(test), we want immediate failure to catch bugs during development.

Q: How do we handle "dust" and non-injected sinks?

Decision: Inequality Check (<=).
Rationale: Strict equality (sum == total) is dangerous for emissions because some TAO might be burned or added to total issuance directly (excess) without being injected into a subnet pool. The invariant total_injected <= block_emission ensures we never create more money than allowed, which is the critical safety property.

3. The Solution Implementation

We implemented a robust invariant enforcement system with the following components:

A. Dedicated Invariant Module (src/invariants.rs)

A new module specifically for consistency checks, separating audit logic from business logic.

  • check_stake_invariant():

    • Iterates all hotkeys in a subnet.
    • Sums TotalHotkeyAlpha.
    • Compares strictly with SubnetAlphaOut.
    • Optimization: Runs only at BlocksSinceLastStep == 0 (epoch boundary) to minimize computational weight.
  • check_emission_invariant():

    • Sums SubnetTaoInEmission across all active subnets.
    • asserts sum(injected) <= BlockEmission.
    • Runs every block to catch inflation bugs immediately.

B. Safety Mechanisms (src/macros/hooks.rs & events.rs)

  • Hook: checks are called at the very end of on_finalize.
  • Event: CriticalInvariantViolation(NetUid, Bytes) alerts the world.
  • Storage: SubnetEmissionPaused map prevents broken subnets from receiving further funds.

C. Recovery Path (src/macros/dispatches.rs)

  • Extrinsic: unpause_subnet_emission(netuid).
  • Permission: Root (Sudo) only.
  • Purpose: Allows governance to fix the underlying state corruption and then resume normal operations manually.

D. Comprehensive Testing (src/tests/invariants.rs)

  • Unit Tests: Verify that:
    1. Normal operation passes.
    2. Stake mismatch triggers panic (in tests).
    3. Emission violation triggers panic (in tests).
    4. Paused subnets are skipped (preventing infinite error loops).
    5. Governance can unpause the subnet.

4. Verification

The changes have been pushed to the fix/invariant-enforcement branch.

  • Git Remote: https://github.com/Dairus01/subtensor
  • Branch: fix/invariant-enforcement

You can verify the fix by running:

cargo test --package pallet-subtensor --lib -- tests::invariants

@Dairus01
Copy link
Author

@open-junius @camfairchild @sam0x17 @gztensor

Please, I would love your review on this PR

@camfairchild
Copy link
Contributor

It would make much more sense to implement something using Imbalances.

Also, running this logic every epoch doesn't seem like a good idea at all.

Update runtime to issue emission as Imbalance. Enforce conservation in subnet emission distribution. Remove redundant invariant checks. Add conservation tests.
@Dairus01
Copy link
Author

It would make much more sense to implement something using Imbalances.

Also, running this logic every epoch doesn't seem like a good idea at all.

Thanks for the feedback. I’ve refactored emission logic to use Substrate Imbalances throughout. Block emission now mints a PositiveImbalance, which is transactionally consumed during subnet distribution, making conservation enforced by construction. The epoch-wide emission invariant has been removed accordingly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants