Add support for fixed size arrays in abi_serializer. #1614

greg7mdp · 2025-06-13T21:08:27Z

Resolves #1401.

Replaces AntelopeIO/leap#1918

action examples

   [[eosio::action]]
   std::array<int,4> testre(std::array<int,4> input){
      std::array<int,4> arr = input;
      for(auto & v : arr) v += 1;
      return arr;
   }

   [[eosio::action]]
   std::vector<int> testrev(std::vector<int> input){
      std::vector<int> vec = input;
      for(auto & v : vec) v += 1;
      return vec;
   }

generated abi (current `cdt` with no change)

...


        {
            "name": "testre",
            "base": "",
            "fields": [
                {
                    "name": "input",
                    "type": "int32[4]"
                }
            ]
        },
        {
            "name": "testrev",
            "base": "",
            "fields": [
                {
                    "name": "input",
                    "type": "int32[]"
                }
            ]
        },
...

    "action_results": [
        {
            "name": "testre",
            "result_type": "int32[4]"
        },
        {
            "name": "testrev",
            "result_type": "int32[]"
        }
    ]

executing the action `testrev` (vector version)

❯ ./bin/cleos --print-response push action hello testrev  '[[1,2,3,4]]' -p hello@active 
...
       "act": {
          "account": "hello",
          "name": "testrev",
          "authorization": [{
              "actor": "hello",
              "permission": "active"
            }
          ],
          "data": {
            "input": [
              1,
              2,
              3,
              4
            ]
          },
          "hex_data": "0401000000020000000300000004000000"
        },
        "context_free": false,
        "elapsed": 48,
        "console": "",
        "trx_id": "c058dc2e87e21e3ffaf7e33544e6d20bc6b30a13276a9439495887f44e9ad4ab",
        "block_num": 1830,
        "block_time": "2025-06-13T21:04:56.500",
        "producer_block_id": null,
        "account_ram_deltas": [],
        "except": null,
        "error_code": null,
        "return_value_hex_data": "0402000000030000000400000005000000",
        "return_value_data": [
          2,
          3,
          4,
          5
        ]
      }
    ],
    "account_ram_delta": null,
    "except": null,
    "error_code": null
  }
}
---------------------
executed transaction: c058dc2e87e21e3ffaf7e33544e6d20bc6b30a13276a9439495887f44e9ad4ab  112 bytes  283 us
#         hello <= hello::testrev               {"input":[1,2,3,4]}
=>                                return value: [2,3,4,5]

executing the action `testre` (fixed size array version)

❯ ./bin/cleos --print-response push action hello testre  '[[1,2,3,4]]' -p hello@active        
...

       "act": {
          "account": "hello",
          "name": "testre",
          "authorization": [{
              "actor": "hello",
              "permission": "active"
            }
          ],
          "data": {
            "input": [
              1,
              2,
              3,
              4
            ]
          },
          "hex_data": "01000000020000000300000004000000"
        },
        "context_free": false,
        "elapsed": 52,
        "console": "",
        "trx_id": "c0fe863f78bd6f0df0d4ad66b0b16818bb41a71dd6ac6764b0f0eabaf92b00c9",
        "block_num": 2043,
        "block_time": "2025-06-13T21:06:43.000",
        "producer_block_id": null,
        "account_ram_deltas": [],
        "except": null,
        "error_code": null,
        "return_value_hex_data": "02000000030000000400000005000000",
        "return_value_data": [
          2,
          3,
          4,
          5
        ]
      }
    ],
    "account_ram_delta": null,
    "except": null,
    "error_code": null
  }
}
---------------------
executed transaction: c0fe863f78bd6f0df0d4ad66b0b16818bb41a71dd6ac6764b0f0eabaf92b00c9  112 bytes  338 us
#         hello <= hello::testre                {"input":[1,2,3,4]}
=>                                return value: [2,3,4,5]

greg7mdp · 2025-06-13T21:16:16Z

Noticed that the type for operator() should probably be fc::unsigned_int , not clear to me if it would make any difference.

spring/libraries/libfc/include/fc/io/varint.hpp

Lines 91 to 99 in 4879600

    
           template<> 
        
           struct hash<fc::unsigned_int> 
        
           { 
        
               public: 
        
                 size_t operator()(const fc::signed_int &a) const  
        
                 { 
        
                    return std::hash<uint32_t>()(a.value); 
        
                 } 
        
           };

heifner · 2025-06-16T18:51:15Z

libraries/libfc/include/fc/variant_object.hpp

+         set(std::move(key), variant( val ) );
+         return std::move(*this);
+      }
      /**


Please add tests for new variant functionality to test_variant.cpp

greg7mdp · 2025-06-18T18:08:05Z

Compatibility with previous versions

Supporting fixed size array did not require any change in cdt, because the std::array type already generated a serialization without an initial size:

https://github.com/AntelopeIO/cdt/blob/259f42785f485dec0690addc3ad8768f43761824/libraries/eosiolib/core/eosio/datastream.hpp#L538-L543

as opposed to std::vector serialization:

https://github.com/AntelopeIO/cdt/blob/259f42785f485dec0690addc3ad8768f43761824/libraries/eosiolib/core/eosio/datastream.hpp#L704-L710

Also, in the abi, std::array was already translated into a fixed size array type (such as "type": "int32[4]"), while std::vector was translated into a variable size array (such as "type": "int32[]").

Spring was lacking support for the fixed size array abi type in abi_serializer::_binary_to_variant() and _variant_to_binary(), which causes cleos to fail when pushing an action that either accepts parameter of type std::array, or returning a value containing such a type:

ex1: passing a std::array as parameter:

❯ ./bin/cleos --print-response push action hello testre  '[[1,2,3,4]]' -p hello@active                                                                                       22.16.0
RESPONSE:
---------------------
{
  "account_name": "hello",
  "code_hash": "0dcfae481ae0d33f3b77a331dba46a7e551acc29320af0ec8c4e72bb2d14ef1d",
  "abi_hash": "835f13b80cafc0fd8298f573cb3c7bb990704fb0c7e529420281251cd9707d09",
  "abi": "DmVvc2lvOjphYmkvMS4yAAgHdGVzdGNvbQABBHVzZXIEbmFtZQZ0ZXN0aW4AAQdtZXNzYWdlBnN0cmluZwZ0ZXN0bmUAAAZ0ZXN0cGEAAQVpbnB1dAhpbnQzMls0XQZ0ZXN0cmUAAQVpbnB1dAhpbnQzMls0XQd0ZXN0cmUyAAEFaW5wdXQHaW50MzJbXQd0ZXN0cmV2AAEFaW5wdXQHaW50MzJbXQV0ZXN0cwACAmlkBnVpbnQ2NANzdHIJdWludDhbMzJdBwAAAEBSlLHKB3Rlc3Rjb20AAAAAAEyXscoGdGVzdGluAAAAAACombHKBnRlc3RuZQAAAAAAmJqxygZ0ZXN0cGEAAAAAAKibscoGdGVzdHJlAAAAAECom7HKB3Rlc3RyZTIAAAAAYKubscoHdGVzdHJldgABAAAAAACcscoDaTY0AAAFdGVzdHMAAAAAAwAAAACom7HKCGludDMyWzRdAAAAQKibscoIaW50MzJbNF0AAABgq5uxygdpbnQzMltd"
}
---------------------
error 2025-06-18T17:33:46.120 cleos     main.cpp:4525                 operator()           ] Failed with error: Bad Cast (7)
Invalid cast from array_type to int64

ex2: action taking a std::vector and returning a std::array:

> ./bin/cleos --print-response push action hello testre2  '[[1,2,3,4]]' -p hello@active  
...
          "hex_data": "0401000000020000000300000004000000"
        },
        "context_free": false,
        "elapsed": 65,
        "console": "",
        "trx_id": "89eecfa239a2f1033ef2404c156e75fee032614e1341ae2dcf810ef887ad8a70",
        "block_num": 24361,
        "block_time": "2025-06-18T17:44:30.000",
        "producer_block_id": null,
        "account_ram_deltas": [],
        "except": null,
        "error_code": null,
        "return_value_hex_data": "02000000030000000400000000000000",
        "return_value_data": 2
      }
    ],
    "account_ram_delta": null,
    "except": null,
    "error_code": null
  }
}
---------------------
executed transaction: 89eecfa239a2f1033ef2404c156e75fee032614e1341ae2dcf810ef887ad8a70  112 bytes  333 us
#         hello <= hello::testre2               {"input":[1,2,3,4]}
=>                                return value: 2

Note that the displayed return value is incorrect (should be [2,3,4,5]).

However, the issue is only in cleos.

I have verified that if the hex data passed to the action ("hex_data": "01000000020000000300000004000000") conforms to the fixed size array abi ("type": "int32[4]") generated by cdt, the action is successfully invoked even in Spring 1.0, and the correct return value is returned ("return_value_hex_data": "02000000030000000400000005000000").

So the changes in this PR only allow cleos to work correctly with actions using std::array in their call interface.
As far as I can tell they do not change the behavior for actions received with the data already serialized.

heifner · 2025-06-18T18:38:02Z

Note that the displayed return value is incorrect (should be [2,3,4,5]).

This is for pre-2.0 versions of cleos, correct? For a cleos built from this PR it will work.

greg7mdp · 2025-06-18T18:52:59Z

This is for pre-2.0 versions of cleos, correct? For a cleos built from this PR it will work.

Yes, exactly. Using the version from this PR all works fine with cleos (passing array parameters, and returning them, whether these are variable or fixed size arrays in the abi).

heifner · 2025-06-18T19:05:19Z

This is for pre-2.0 versions of cleos, correct? For a cleos built from this PR it will work.

Yes, exactly. Using the version from this PR all works fine with cleos (passing array parameters, and returning them, whether these are variable or fixed size arrays in the abi).

Presumably it is the version of nodeos that matters here not cleos as it is JSON from cleos perspective.

heifner · 2025-06-18T19:08:19Z

Presumably it is the version of nodeos that matters here not cleos as it is JSON from cleos perspective.

Actually it will be version of cleos for action args and nodeos for action return values.

greg7mdp · 2025-06-18T19:39:25Z

Presumably it is the version of nodeos that matters here not cleos as it is JSON from cleos perspective.

Actually it will be version of cleos for action args and nodeos for action return values.

I think it is always the version of cleos used which matters, since I think the abi_serializer code is linked into cleos. Even spring 1.0 returns the correct return hex value (since it is created by the wasm code compiled from cdt, which has supported fixed size arrays for a while).

heifner · 2025-06-18T19:47:28Z

Presumably it is the version of nodeos that matters here not cleos as it is JSON from cleos perspective.

Actually it will be version of cleos for action args and nodeos for action return values.

I think it is always the version of cleos used which matters, since I think the abi_serializer code is linked into cleos. Even spring 1.0 returns the correct return hex value (since it is created by the wasm code compiled from cdt, which has supported fixed size arrays for a while).

The transaction_trace is abi de-serialized in nodeos not cleos. All cleos does is convert the JSON/variant to a pretty-string without any interaction with abi_serializer.
The action payload is abi serialized in cleos.

…izer_list`

greg7mdp · 2025-06-20T13:35:09Z

After discussion in standup, rebasing this PR to main as it only fixes bugs in binary <-> variant when std::array is used in contracts (and cdt generates an T[sz] abi type), but does not change anything that was not broken.

spoonincode · 2025-06-24T16:15:20Z

unittests/abi_tests.cpp

+           "base": "",
+           "fields": [{
+               "name": "a",
+               "type": "uint8[5]"


it looks like uint8[]$ is considered a valid type currently, so probably should have a test for something like uint8[5]$. Should uint8[5]? be allowed? I'd say make either a positive or negative test for that too depending on what we want (it kinda feels like it ought to be allowed but I'm not sure I have an immediate strong opinion on it).

Yes, I think it should be allowed as well. I'll add a test for it.

Well, actually, neither abi_serializer nor abieos seem to support optional vectors (var size arrays).
It doesn't seem like it would be too hard to fix though.

However, to do it properly would require some kind of recursion though, as a type can be T?, and T can be T2[], and T2 can be T3?, so we cannot expect to get to the fundamental_type without recursion as we do today in:

spring/libraries/chain/abi_serializer.cpp

Lines 252 to 262 in 5227aa3

std::string_view abi_serializer::fundamental_type(const std::string_view& type)const {

if( is_array(type) ) {

return type.substr(0, type.size()-2);

} else if (is_szarray (type) ){

return type.substr(0, type.find_last_of('['));

} else if ( is_optional(type) ) {

return type.substr(0, type.size()-1);

} else {

return type;

}

}

If we want to do that I think it probably should be in a separate PR.

What about uint8[5]$ though? Pretty sure uint8[]$ works today (at least, there looks like a test for it).

…not match.

spoonincode

I'd recommend updating the wiki doc specifying that only type[n]$ is allowed, not type[n]?. Maintainers of other implementations might have the same uncertainty during development.

I still think it would be useful to have a test that verifies type[n]? is sanely rejected and doesn't do anything funky.

But it wasn't clear to me if you were intending to tackle type[n]? later anyways. So maybe the 2 comments above doesn't apply

…) are rejected by the abi.

greg7mdp · 2025-06-25T18:25:17Z

I added a sentence in the limitiations of the fixed size array wiki: https://github.com/AntelopeIO/spring/wiki/ABI-1.3:-Fixed-sized-arrays.

Also added tests for type[n]? and type[]?.

I'm happy to add support for type[n]? and type[]? in a separate PR if you think it is worthwhile. I think it may be a better goal to support all type combinations (such as int8[3]?[] for a vector of optional triplets).

spoonincode · 2025-06-25T19:36:26Z

oh don't forget to add [2.0.0-dev] to the PR

I'm happy to add support for type[n]? and type[]? in a separate PR if you think it is worthwhile

idk if the juice is worth the squeeze. I was more interested in us being clear what was and was not expected to work

spoonincode · 2025-06-25T19:37:50Z

oh don't forget to add [2.0.0-dev] to the PR

actually wait, why is this going to main right now?

greg7mdp · 2025-06-25T19:43:01Z

actually wait, why is this going to main right now?

We talked about it in standup last week, and Areg asked me to move it back to main because we chose to consider that cdt had correct support for a while, and the abi_serializer not supporting it was a bug. Since it was not working before, it was expected that people were not using fixed size arrays, so it is not a breaking change to make them work.

BTW don't know if you noticed, warfkit also fixed their support for fixed size arrays wharfkit/antelope#119

spoonincode · 2025-06-25T19:44:52Z

I completely disagree but I'm not going to debate it any more

greg7mdp mentioned this pull request Jun 13, 2025

Support serialization of fixed-size arrays in abi_serializer. AntelopeIO/leap#1918

Closed

heifner approved these changes Jun 16, 2025

View reviewed changes

greg7mdp requested a review from spoonincode June 16, 2025 12:49

heifner reviewed Jun 16, 2025

View reviewed changes

greg7mdp mentioned this pull request Jun 16, 2025

Add test for array/fixed size array support in action data and action return value. AntelopeIO/cdt#358

Merged

heifner approved these changes Jun 17, 2025

View reviewed changes

heifner mentioned this pull request Jun 18, 2025

Add support for fixed size arrays wharfkit/antelope#118

Closed

greg7mdp added 9 commits June 20, 2025 08:49

Add support for fixed-size arrays in abi.

45d2203

Add check for array length.

d4a8ced

Remove commented out code.

409c57e

Remove unnecessary memory alloc (immediately overwritten)

69f12c2

Add support for initializing fc::variant array with a `std::initial…

9687e06

…izer_list`

Take initializer_list as a const&, and add constructor.

d81aaff

rebase to main

afd9317

Fix merge issue.

6f6dfed

Merge branch 'main' of github.com:AntelopeIO/spring into gh_1401

00b5a37

greg7mdp force-pushed the gh_1401 branch from ae198d3 to 00b5a37 Compare June 20, 2025 13:29

greg7mdp changed the base branch from release/2.0 to main June 20, 2025 13:30

Whitespace change.

1ac3fd5

heifner approved these changes Jun 20, 2025

View reviewed changes

greg7mdp requested a review from linh2931 June 23, 2025 20:33

spoonincode reviewed Jun 24, 2025

View reviewed changes

greg7mdp added 5 commits June 24, 2025 14:23

Merge branch 'main' of github.com:AntelopeIO/spring into gh_1401

5227aa3

Merge branch 'main' of github.com:AntelopeIO/spring into gh_1401

84e1a7c

Add test to verify that fixed array binary extension is supported.

ca9c741

Merge branch 'main' of github.com:AntelopeIO/spring into gh_1401

f66314b

Make identification of fixed size arrays stricter. int8[5]? should …

3f35db2

…not match.

spoonincode approved these changes Jun 25, 2025

View reviewed changes

Add test cases to verify that optional arrays (variable or fixed size…

0cfff9c

…) are rejected by the abi.

greg7mdp merged commit 2d3b5b4 into main Jun 25, 2025
36 checks passed

greg7mdp deleted the gh_1401 branch June 25, 2025 19:37

	std::string_view abi_serializer::fundamental_type(const std::string_view& type)const {
	if( is_array(type) ) {
	return type.substr(0, type.size()-2);
	} else if (is_szarray (type) ){
	return type.substr(0, type.find_last_of('['));
	} else if ( is_optional(type) ) {
	return type.substr(0, type.size()-1);
	} else {
	return type;
	}
	}

Add support for fixed size arrays in abi_serializer. #1614

Add support for fixed size arrays in abi_serializer. #1614

Uh oh!

Conversation

greg7mdp commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

action examples

generated abi (current cdt with no change)

executing the action testrev (vector version)

executing the action testre (fixed size array version)

Uh oh!

greg7mdp commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

heifner Jun 16, 2025

Choose a reason for hiding this comment

Uh oh!

greg7mdp Jun 17, 2025

Choose a reason for hiding this comment

Uh oh!

greg7mdp commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Compatibility with previous versions

Uh oh!

heifner commented Jun 18, 2025

Uh oh!

greg7mdp commented Jun 18, 2025

Uh oh!

heifner commented Jun 18, 2025

Uh oh!

heifner commented Jun 18, 2025

Uh oh!

greg7mdp commented Jun 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

heifner commented Jun 18, 2025

Uh oh!

greg7mdp commented Jun 20, 2025

Uh oh!

spoonincode Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

greg7mdp Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

greg7mdp Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

spoonincode Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

greg7mdp Jun 24, 2025

Choose a reason for hiding this comment

Uh oh!

spoonincode left a comment

Choose a reason for hiding this comment

Uh oh!

greg7mdp commented Jun 25, 2025

Uh oh!

spoonincode commented Jun 25, 2025

Uh oh!

spoonincode commented Jun 25, 2025

Uh oh!

Uh oh!

greg7mdp commented Jun 25, 2025

Uh oh!

spoonincode commented Jun 25, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

greg7mdp commented Jun 13, 2025 •

edited

Loading

generated abi (current `cdt` with no change)

executing the action `testrev` (vector version)

executing the action `testre` (fixed size array version)

greg7mdp commented Jun 13, 2025 •

edited

Loading

greg7mdp commented Jun 18, 2025 •

edited

Loading

greg7mdp commented Jun 18, 2025 •

edited

Loading