Skip to content

add encoder for list when using native types#1097

Open
darshanime-d11 wants to merge 4 commits intoSFDO-Tooling:mainfrom
darshanime-d11:list_v3
Open

add encoder for list when using native types#1097
darshanime-d11 wants to merge 4 commits intoSFDO-Tooling:mainfrom
darshanime-d11:list_v3

Conversation

@darshanime-d11
Copy link

currently when using native types (when snowfakery_version: 3) is specified, this example recipe from docs fails:

- snowfakery_version: 3
- object: SomeObject
  fields:
    formula_field_example: ${{fake.json(data_columns={'Spec':'@1.0.1',
      'ID':'pyint','Details':{'Name':'name', 'Address':'address'}},
      num_rows=10)}}

this pr adds an encoder for list that generates a valid json that is backwards compatible with version 2

$ cat SomeObject.csv
formula_field_example,float_field,id
"[{""Spec"": ""1.0.1"", ""ID"": 6243, ""Details"": {""Name"": ""Dr. Kyle Jarvis Jr."", ""Address"": ""USNS English\nFPO AE 13563""}}, {""Spec"": ""1.0.1"", ""ID"": 1889, ""Details"": {""Name"": ""Ashley Black DDS"", ""Address"": ""USNV Mcknight\nFPO AA 10708""}}, {""Spec"": ""1.0.1"", ""ID"": 738, ""Details"": {""Name"": ""Isabella Haynes PhD"", ""Address"": ""Unit 1410 Box 9209\nDPO AE 28376""}}, {""Spec"": ""1.0.1"", ""ID"": 2713, ""Details"": {""Name"": ""Lydia Rubio MD"", ""Address"": ""PSC 8156, Box 6235\nAPO AA 56803""}}, {""Spec"": ""1.0.1"", ""ID"": 3728, ""Details"": {""Name"": ""Jay Key II"", ""Address"": ""PSC 3947, Box 8794\nAPO AE 45672""}}, {""Spec"": ""1.0.1"", ""ID"": 2060, ""Details"": {""Name"": ""David Owen"", ""Address"": ""4575 George Drives\nLake Lee, KS 03067""}}, {""Spec"": ""1.0.1"", ""ID"": 9155, ""Details"": {""Name"": ""Dr. Colin Willis"", ""Address"": ""48809 Hull Groves Apt. 854\nSouth Parkerfurt, AK 50804""}}, {""Spec"": ""1.0.1"", ""ID"": 8396, ""Details"": {""Name"": ""Mrs. Nicole Banks"", ""Address"": ""Unit 5502 Box 3002\nDPO AP 52724""}}, {""Spec"": ""1.0.1"", ""ID"": 1926, ""Details"": {""Name"": ""Dr. Fernando Hull"", ""Address"": ""USNS Finley\nFPO AE 91106""}}, {""Spec"": ""1.0.1"", ""ID"": 8764, ""Details"": {""Name"": ""Mr. Anthony Fox Jr."", ""Address"": ""PSC 2680, Box 3112\nAPO AA 36340""}}]",2.2,1

@darshanime-d11 darshanime-d11 requested a review from a team as a code owner March 18, 2025 13:39
@salesforce-cla
Copy link

Thanks for the contribution! Before we can merge this, we need @darshanime-d11 to sign the Salesforce Inc. Contributor License Agreement.

@darshanime-d11
Copy link
Author

can you please review this PR @jstvz, @vsbharath

@jstvz jstvz self-assigned this Apr 8, 2025
Copy link
Contributor

@jstvz jstvz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the PR, @darshanime-d11. One change requested.

@darshanime-d11
Copy link
Author

i have added the additional types and encoders for them.

sample output:

$ cat /tmp/foo.yaml
- snowfakery_version: 3
- object: foo
  fields:
    a: 1
    c: 3
- object: foo
  fields:
    b: 2
    d: 4
- object: bar
  fields:
    barb: 2
    bard: 4
- object: faz
  fields:
    list_t: ${{fake.json(data_columns={'Spec':'@1.0.1', 'ID':'pyint','Details':{'Name':'name', 'Address':'address'}}, num_rows=2)}}
    dict_t: ${{fake.json(data_columns={'Spec':'@1.0.1', 'ID':'pyint','Details':{'Name':'name', 'Address':'address'}}, num_rows=1)}}
    bool_t: ${{fake.pybool()}}
    tuple_t: ${{(1, 2)}}
    bytes_t: ${{fake.binary(length=64)}}



$ ./venv2/bin/snowfakery /tmp/foo.yaml --output-format csv && cat faz.csv
Created foo.csv
Created bar.csv
Created faz.csv
Created csvw_metadata.json
list_t,dict_t,bool_t,tuple_t,bytes_t,id
"[{""Spec"": ""1.0.1"", ""ID"": 7680, ""Details"": {""Name"": ""Mr. Roberto Gentry"", ""Address"": ""Unit 7523 Box 4996\nDPO AE 41113""}}, {""Spec"": ""1.0.1"", ""ID"": 1399, ""Details"": {""Name"": ""Luke Oconnor"", ""Address"": ""471 Stacey Lakes\nLorimouth, AS 83846""}}]","{""Spec"": ""1.0.1"", ""ID"": 5858, ""Details"": {""Name"": ""Colin Harrington PhD"", ""Address"": ""Unit 6316 Box 3475\nDPO AP 83921""}}",0,"(1, 2)",b'\xbc\xe6\x07\xf3\xdd\xe8p~E~\x1e\x93\x02^r\xbf&\xe0\xd4MB\x95\xa6\xb7\xef:\xf6\xc9\x91\xdb\xa4\x85!CP\x05\xe2\x7f\x965\xc1k\xafU\x11K\xa8\t\x82\\\x9f\x9f\xd7\xaa\x84M\x858\xcev{a\xbbm',1

@darshanime-d11
Copy link
Author

ptal @jstvz, @vsbharath

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants