Respect the schema passed in the stream.

[cgardens]

• Extract headers from schema object and use those headers to write to the CSV.

• Write tests that flatten and get_headers produce the same keys.

Description of change

Currently this integration does not use schema to set headers in the CSV files. It instead relies on what is already in the file or the keys it can find in the record. I believe this is the source of these issues #3 and #22. My understanding of the singer spec is that it would be appropriate to use the schema records to make sure the correct headers are used in the CSV files, instead of sampling records.

Manual QA steps

• Ran against the exchangeratesapi tap to sanity check that everything still worked. Wrote tests to check the specific behavior around missing headers. Happy to do more if this PR gains traction.

Risks

• This change should be save, because if the record does not comply with the schema, the integration throws an error already. It will change the output in the cases where sampling the record was leading to incomplete headers, so it is technically a breaking change, but I think it is only breaking in cases that were already pretty broken.

Rollback steps

• revert this branch

[cmerrick]

Hi @cgardens, thanks for your contribution!

In order for us to evaluate and accept your PR, we ask that you sign a contribution license agreement. It's all electronic and will take just minutes.

[cmerrick]

You did it @cgardens!

Thank you for signing the Singer Contribution License Agreement.