docs(parquet): add example for preserving dictionary encoding

[AndreaBozzo]

This PR adds a second example to ArrowReaderOptions::with_schema demonstrating how to preserve dictionary encoding when reading Parquet string columns.

Closes #9095

[mhilton]

Thanks for the added docs. This reads well and makes sense to me, someone with more understanding will have to decide if the code is correct (I assume it is).

[AndreaBozzo]

Thanks for the added docs. This reads well and makes sense to me, someone with more understanding will have to decide if the code is correct (I assume it is).

Thank you for your time !

[alamb]

Thanks @AndreaBozzo -- this looks great

The only thing I want to resolve before merge is the Notes section. All the other comments are nice to haves / would be good to do as follow on PRs

Also thanks @mhilton for the review

[alamb]

I think this is somewhat misleading beacuse

1. A single read never spans multiple column chunks
2. the batch size is not related to the dictionary size, as far as I know

Dictionary encoding works best when:

1. The original column was dictionary encoded (the default)
2. There are a small number of distinct values

[alamb]

I think checking the schema type and then the actual record batch type is redundant -- you could leave out this check and make the example more concise

[alamb]

It would be nice to prefix the setup of the example with # so it isn't rendered in the docs

So instead of

    /// use tempfile::tempfile;

Do

    /// # use tempfile::tempfile;

That still runs, but will not be shown

[alamb]

However, given this example follows the others in this, file, I think it is fine to keep it this way and we can improve all the examples as a follow on PR

[alamb]

I also see that this follows the example above -- I think we could make these examples smaller (and thus easier to follow) if we wrote into an in memory Vec rather than a file

let mut file = Vec::new();
...
let mut writer = ArrowWriter::try_new(&mut file, batch.schema(), None).unwrap();
writer.write(&batch).unwrap();
writer.close().unwrap();
...
// now read from the "file"

Since what you have here is following the same pattern of the other examples, I think it is good. Maybe we can improve the examples as a follow on PR

[AndreaBozzo]

Thank you for the thorough review and your time! I addressed your feedback in commit 1ac4d54:

if the follow up about the fourth pattern is needed, i/we can handle that aswell

[alamb]

Thanks @AndreaBozzo

[alamb]

if the follow up about the fourth pattern is needed, i/we can handle that aswell

I am not quite sure what you mean by this

[AndreaBozzo]

if the follow up about the fourth pattern is needed, i/we can handle that aswell

I am not quite sure what you mean by this

Sorry i meant i can improve the examples if needed

[alamb]

if the follow up about the fourth pattern is needed, i/we can handle that aswell

I am not quite sure what you mean by this

Sorry i meant i can improve the examples if needed

That would be great if you have the time -- specifically I think applying both these comments to all the examples would improve the readability 🙏

• docs(parquet): add example for preserving dictionary encoding #9116 (comment)
• docs(parquet): add example for preserving dictionary encoding #9116 (comment)

[alamb]

Thanks again @AndreaBozzo