Account data not deserialized when deploying native program with Shank IDL

[santicevic]

Issue: I’m deploying a native Solana program via a Surfpool runbook and explicitly providing a Shank-generated IDL. The runbook appears to load and parse the IDL correctly (e.g. intentionally breaking the IDL schema causes Surfpool to error as expected).
However, after invoking an instruction on the deployed program, Surfpool Studio does not deserialize account data. Instead, the account data is shown only as a base64 string in the Pretty tab.

Versions:
shank 0.4.6
solana-program 2.3.0

Steps to reproduce:

1. Deploy native program via runbook

action "deploy_seedplex" "svm::deploy_program" {
    description = "Deploy seedplex program"
    program = svm::get_program_from_native_project("seedplex", "./target/deploy/seedplex-keypair.json", "./idl/seedplex.json") 
    authority = signer.authority
    payer = signer.payer
}

2. IDL (seedplex.json)
   Below is a shortened version of the IDL containing a single instruction:

{
  "version": "0.1.0",
  "name": "seedplex",
  "instructions": [
    {
      "name": "InitSeedplexDao",
      "accounts": [
        {
          "name": "seedplexAdmin",
          "isMut": true,
          "isSigner": true,
          "docs": [
            "Seedplex admin"
          ]
        },
        {
          "name": "seedplexDao",
          "isMut": true,
          "isSigner": false,
          "docs": [
            "Seedplex DAO"
          ]
        },
        {
          "name": "adminRecord",
          "isMut": true,
          "isSigner": false,
          "docs": [
            "Admin Record"
          ]
        },
        {
          "name": "systemProgram",
          "isMut": false,
          "isSigner": false,
          "docs": [
            "System Program Account"
          ]
        }
      ],
      "args": [
        {
          "name": "initSeedplexDaoArgs",
          "type": {
            "defined": "InitSeedplexDaoArgs"
          }
        }
      ],
      "discriminant": {
        "type": "u8",
        "value": 0
      }
    }
  ],
  "accounts": [
    {
      "name": "AdminRecord",
      "type": {
        "kind": "struct",
        "fields": [
          {
            "name": "key",
            "type": {
              "defined": "Key"
            }
          },
          {
            "name": "wallet",
            "type": "publicKey"
          },
          {
            "name": "bump",
            "type": "u8"
          }
        ]
      }
    },
    {
      "name": "SeedplexDao",
      "type": {
        "kind": "struct",
        "fields": [
          {
            "name": "key",
            "type": {
              "defined": "Key"
            }
          },
          {
            "name": "adminMultisig",
            "type": "publicKey"
          },
          {
            "name": "bump",
            "type": "u8"
          }
        ]
      }
    }
  ],
  "types": [
    {
      "name": "InitSeedplexDaoArgs",
      "type": {
        "kind": "struct",
        "fields": [
          {
            "name": "adminMultisig",
            "type": "publicKey"
          }
        ]
      }
    },
    {
      "name": "Key",
      "type": {
        "kind": "enum",
        "variants": [
          {
            "name": "Uninitialized"
          },
          {
            "name": "SeedplexDao"
          }
        ]
      }
    }
  ],
  "errors": [],
  "metadata": {
    "origin": "shank",
    "address": "sEEDQ9EnkDQGgyvuzMWJrMdnxEGzhvrNFKvLFEPufMD"
  }
}

3. Invoke instruction
   Invoke the InitSeedplexDao instruction.

Actual Behavior
After invocation, Surfpool Studio does not deserialize the program accounts. In the Pretty tab, the account data is displayed only as a base64 string rather than a decoded struct based on the IDL.

Expected Behavior
Account data should be deserialized and displayed according to the IDL definitions (AdminRecord, SeedplexDao) when viewing the account in Surfpool Studio.

Surfpool studio after invoking the InitSeedplexDao instruction

[kimberly75544-netizen]

@santicevic has this issue been resolved?

[MicaiahReid]

Thanks for the detailed issue, @santicevic!

This function is what's encoding accounts with the IDL:

surfpool/crates/core/src/surfnet/svm.rs

Line 2369 in 5c56817

pub fn encode_ui_account<T: ReadableAccount>(

Currently there are a few assumptions that we're making, which we could refine to be more robust. Most of these assumptions are based on how anchor is generating IDLs. We're assuming:

• the first 8 bytes make up the discriminator
• the account data is borsh encoded

I have a hunch that the first one is not being met here. Your IDL doesn't include any discriminator for the accounts, so we're assuming it's an anchor-style discriminator - an the first 8 bytes of the sha hash of "account" + account.name. If your program has a different discriminator, we won't be able to match the account data to the expected account in the IDL.

If your program's accounts are using an 8-byte discriminator that is calculated differently, you should just be able to add a discriminator field to your IDL's accounts and it should work. If your discriminator is less than 8 bytes, surfpool will need to change some code (which is fine, this is a needed improvement).

Can you let me know how your accounts are identified, or do you have a repo you can give me access to so I can try to figure this out?

[santicevic]

Hey @MicaiahReid, really appreciate the fast and detailed reply, that makes sense 👍

You’re right about the discriminator assumption. In this program the accounts use a single-byte discriminator, not an Anchor-style 8-byte one. The first byte of the account is an enum value that identifies the account type.

#[derive(
    Clone, Copy, BorshSerialize, BorshDeserialize, Debug, PartialEq, Eq, ToPrimitive, FromPrimitive,
)]
pub enum Key {
    Uninitialized,
    SeedplexDao
}

That enum is the first field in the account and effectively serves as the discriminator:

#[derive(BorshSerialize, BorshDeserialize, Debug, ShankAccount)]
pub struct SeedplexDao {
    pub key: Key,
    pub admin_multisig: Pubkey,
    pub bump: u8,
}

impl SeedplexDao {
    pub const SPACE: usize = 1 + 32 + 1;
    pub fn new(admin_multisig: Pubkey, bump: u8) -> Self {
        Self {
            key: Key::SeedplexDao,
            admin_multisig,
            bump,
        }
    }
}

So yeah, that would explain why Surfpool can’t currently match the account data to the IDL, the discriminator is only 1 byte and doesn’t follow the Anchor convention.

I can update the program to use an 8-byte Anchor-style discriminator if needed, but if you’re already planning to make Surfpool more flexible here, I’m also happy to leave it as-is and wait for the fix on your side. Just let me know what you’d prefer.

Thanks again!

[MicaiahReid]

Yo @santicevic! I think I have a fix for you. Can you build surfpool on this branch? #451

Additionally, you can update your IDL to manually add a discriminator for your accounts. If you leave it blank, it will automatically to be computed to be the 8-byte version. Instead, you'll need to provide just the one byte in an array.

I think with these two together, your account should be properly parsed! Lmk if you need additional help or instruction, or if things still don't work.

[santicevic]

Hey @MicaiahReid, thanks again for the super fast turnaround on this 🙏

I gave it a try, but unfortunately the accounts are still not getting deserialized in Studio.

I updated the IDL to explicitly include single-byte discriminators on the accounts. This is what the relevant section of the IDL looks like now:

  "accounts": [
    {
      "discriminator": [1],
      "name": "SeedplexDao",
      "type": {
        "kind": "struct",
        "fields": [
          {
            "name": "key",
            "type": {
              "defined": "Key"
            }
          },
          {
            "name": "adminMultisig",
            "type": "publicKey"
          },
          {
            "name": "bump",
            "type": "u8"
          }
        ]
      }
    },
    {
      "discriminator": [2],
      "name": "AdminRecord",
      "type": {
        "kind": "struct",
        "fields": [
          {
            "name": "key",
            "type": {
              "defined": "Key"
            }
          },
          {
            "name": "wallet",
            "type": "publicKey"
          },
          {
            "name": "bump",
            "type": "u8"
          }
        ]
      }
    },

I built Surfpool from the fix/variable-len-discriminator branch and confirmed I’m running that binary, but I’m still seeing the same behavior in Studio (account data shown as base64, no deserialization).

Just to double-check the on-chain layout, here’s a screenshot of the raw hex data from Studio showing that the account data does indeed start with a single-byte discriminator:

Let me know if there’s anything else I can do to help debug this, happy to add logging or try another branch.

Thanks again!

[MicaiahReid]

Hey @santicevic is there any chance you can add me to the repo to take a look at this?

[santicevic]

Hey @MicaiahReid we just gave you read access to the repo.

There is a codama rust and ts client you can use to invoke the InitSeedplexDao instruction. Let me know if I can help with anything.

[MicaiahReid]

Great, thanks for the access, @santicevic! I've found a few issues in how we're trying to upgrade the shank IDL to an anchor IDL. I'll need some time to work through this, but it should be achievable! I'll keep you updated.

[santicevic]

@MicaiahReid perfect, thanks for the update 🙌 no rush at all from our side. Appreciate you taking a look.