[fix][test] Fixed Nondeterministic Ordering in Generated Docs Topics

[LucasEby]

Fixes apache#24968

Motivation

The below test compares the generated documentation of CmdGenerateDocsTest#testGenerateDocs. The test assumes a deterministic order of the generated documentation but the order of the listed help and name options was not guaranteed. Due to two sources of nondeterminism hidden in the test and production code, the options could be listed in a different order. Picocli's CommandLine class uses Class#getDeclaredFields internally (via reflection), which has a non-deterministic specification to retrieve the Option annotations and orders these options accordingly. Even though picocli supports an order attribute on Option annotations, CommandLine never sorts by this order - it just maintains whatever order the reflection API gave it. CmdGenerateDocs#generateDocument then iterates over the order that the args method returns, with the assumption that the order is deterministic. Unfortunately, the ordering can change due to different environments producing the contents in different orders despite the logical contents being the same. Harmless re-ordering could flip the tests from pass to fail despite the data being semantically the same.

• org.apache.pulsar.docs.tools.CmdGenerateDocsTest.testGenerateDocs

Modifications

The order of each option in the Argument class is now explicitly stated so getDeclaredFields is no longer utilized to determine the order of the options. Additionally, generateDocument now sorts all argSpecs by their specified order with a new helper method called getSortedArgs. This combination of changes enforces the expected nondeterminism in the ordering of the listed options.

If order is not specified because it does not matter, the order for each option will remain in it's original order as before. This is because according to Collection's documentation, it's sort method is guaranteed to be stable: equal elements will not be reordered as a result of the sort (and the orders are equal because Piccocli's CommandLine class sets order to -1 by default).

Verifying this change

• Make sure that the change passes the CI checks.

This change is already covered by existing tests, such as

• org.apache.pulsar.client.impl.schema.SchemaInfoTest$SchemaInfoBuilderTest.testNullPropertyValue
• org.apache.pulsar.client.impl.schema.SchemaInfoTest.testSchemaInfoToString

Does this pull request potentially affect one of the following parts:

If the box was checked, please highlight the changes

• Dependencies (add or upgrade a dependency)
• The public API
• The schema
• The default values of configurations
• The threading model
• The binary protocol
• The REST endpoints
• The admin CLI options
• The metrics
• Anything that affects deployment

Documentation

• doc
• doc-required
• doc-not-needed
• doc-complete

Matching PR in forked repository

PR in forked repository:

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.

📢 Thoughts on this report? Let us know!