feat(voting): add configurable tie-breaker strategy

[WangCheng0116]

Description

This PR includes the first two commits on voting-tiebreaker and introduces
configurable tie-breaking for VotingEvaluationStrategy, plus corresponding
unit test coverage.

Background and purpose

VotingEvaluationStrategy had a fixed tie resolution behavior. This PR makes
tie-breaking configurable and verifies each supported strategy and error path.

Changes made

• Added configurable tie_breaker support in VotingEvaluationStrategy:
  • min
  • max
  • mean_closest
  • custom callable
• Refactored tie-breaker validation to remove duplicated checks.
• Added tests for:
  • valid/invalid tie_breaker initialization
  • tie behavior for each built-in strategy
  • custom callable behavior and invalid callable return value

Included commits

• 8077a0e feat(voting): add configurable tie-breaker
• 0ee4f24 test(voting): add tie-breaker strategy tests

Checklist

Please check the following items before code is ready to be reviewed.

• Code has been formatted with pre-commit run --all-files command
• All tests are passing
• Docstrings are in Google style
• Related documentation has been updated (e.g. links, examples, etc.)
• Code is ready for review

[gemini-code-assist]

Summary of Changes

Hello @WangCheng0116, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the VotingEvaluationStrategy by introducing a configurable tie-breaking mechanism. Previously, ties were resolved using a fixed lowest-score selection. Now, users can specify how ties should be handled, choosing from predefined strategies like min, max, mean_closest, or providing a custom callable function, thereby increasing the flexibility and robustness of the evaluation process.

Highlights

• Configurable Tie-Breaking: Introduced configurable tie-breaking strategies (min, max, mean_closest, or custom callable) for VotingEvaluationStrategy to resolve ties among most frequent results.
• Refactored Tie Resolution: Refactored the VotingEvaluationStrategy to include a dedicated _resolve_tie method, centralizing the logic for handling tie resolution based on the configured strategy.
• Comprehensive Test Coverage: Expanded unit test coverage to validate various tie-breaker configurations, including valid/invalid initialization, behavior of built-in strategies, and custom callable functionality with error path checks.

Changelog

• openjudge/evaluation_strategy/voting_evaluation_strategy.py
  • Added a SUPPORTED_TIE_BREAKERS set to define valid string-based tie-breaker options.
  • Modified the __init__ method to accept a tie_breaker argument, which can be a string or a callable, and added validation for this parameter.
  • Implemented a new private method _resolve_tie to encapsulate the logic for resolving ties based on the configured strategy.
  • Updated the execute method to utilize the new _resolve_tie method when a tie among most frequent results occurs, replacing the previous hardcoded min selection.
  • Updated docstrings for the class, __init__, and execute methods to reflect the new tie_breaker parameter and its behavior.
• tests/evaluation_strategy/test_voting_evaluation_strategy.py
  • Added new test cases to verify successful initialization of VotingEvaluationStrategy with valid string and callable tie_breaker parameters.
  • Included tests to ensure ValueError is raised for invalid tie_breaker inputs during initialization.
  • Introduced dedicated asynchronous test methods to verify the correct tie-breaking behavior for min, max, and mean_closest strategies.
  • Added a test for a custom callable tie-breaker, ensuring it correctly resolves ties.
  • Included a test case to confirm that a ValueError is raised if a custom callable tie-breaker returns a value not present in the tie candidates.

Activity

• Initial commit 8077a0e introduced the configurable tie-breaker feature.
• Second commit 0ee4f24 added comprehensive unit tests for the tie-breaker strategies.
• Docstrings have been updated to Google style.
• The code is marked as ready for review.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	/gemini review	Performs a code review for the current pull request in its current state.
Pull Request Summary	/gemini summary	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	/gemini help	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

[WangCheng0116]

i see there is a todo in voting_evaluation_strategy.py, so i implement it here. Any feedback is appreciated!

[gemini-code-assist]

Code Review

This pull request introduces a configurable tie-breaking mechanism for the VotingEvaluationStrategy. The implementation is solid with comprehensive test coverage, and no security vulnerabilities were identified. Suggestions for further enhancement include making an error message more robust in the main implementation and reducing test code duplication by parameterizing several test cases.

[WangCheng0116]

ready for review @helloml0326, thanks in advance!

[ployts]

Thank you for the implementation—it looks good to me. @helloml0326 , could you please review it again?

[WangCheng0116]

@weizhang25 updated