Fix QAEvaluator to pass through is_reasoning_model parameter

[Copilot]

Description

QAEvaluator was not forwarding the is_reasoning_model parameter to its underlying evaluators, causing failures with reasoning models (o1/o3) that don't support max_tokens:

# Before: fails with reasoning models
qa_eval = QAEvaluator(model_config=config, is_reasoning_model=True)
# Error: "Unsupported parameter: 'max_tokens' is not supported with this model"

Changes

• QAEvaluator: Added is_reasoning_model: bool = False parameter and passed it through to all model-based evaluators (GroundednessEvaluator, RelevanceEvaluator, CoherenceEvaluator, FluencyEvaluator, SimilarityEvaluator)
• Tests: Added unit tests verifying parameter propagation with both default and explicit values
• Formatting: Applied black formatting to evaluator instantiations and removed trailing whitespace

F1ScoreEvaluator unchanged as it performs local computation without LLM calls.

Usage

qa_eval = QAEvaluator(
    model_config=model_config,
    is_reasoning_model=True  # Now properly forwarded to sub-evaluators
)

All SDK Contribution checklist:

• The pull request does not introduce [breaking changes]
• CHANGELOG is updated for new features, bug fixes or other significant changes.
• I have read the contribution guidelines.

General Guidelines and Best Practices

• Title of the pull request is clear and informative.
• There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

• Pull request includes test coverage for the included changes.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>QAEvaluator does not pass through is_reasoning_model</issue_title>
<issue_description>I was trying to use the QAEvaluator w/ a reasoning model w/ the is_reasoning_model: true parameter and ran into the following error:

openai.BadRequestError: Error code: 400 - {'error': {'message': "Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead.", 'type': 'invalid_request_error', 'param': 'max_tokens', 'code': 'unsupported_parameter'}}

This doesn't happen for individual evaluators that the QAEvaluator makes use of. It seems to be due to the fact that the QAEvaluator does not pass through the is_reasoning_model for the evaluators under the hood > https://github.com/Azure/azure-sdk-for-python/blob/main/sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_qa/_qa.py#L104-L111</issue_description>

Comments on the Issue (you are @copilot in this section)

• Fixes QAEvaluator does not pass through is_reasoning_model #43842

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.

[Copilot]

Pull request overview

This PR fixes a bug where QAEvaluator was not forwarding the is_reasoning_model parameter to its underlying model-based evaluators, causing failures when used with reasoning models (o1/o3) that don't support the max_tokens parameter.

Key changes:

• Added is_reasoning_model parameter to QAEvaluator.__init__() with default value False
• Forwarded the parameter to all five model-based sub-evaluators (GroundednessEvaluator, RelevanceEvaluator, CoherenceEvaluator, FluencyEvaluator, SimilarityEvaluator)
• Added unit tests verifying proper parameter propagation for both default and explicit values

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/_evaluators/_qa/_qa.py	Added is_reasoning_model parameter and forwarded it to all model-based sub-evaluators, excluding F1ScoreEvaluator which performs local computation
sdk/evaluation/azure-ai-evaluation/tests/unittests/test_built_in_evaluator.py	Added two test cases verifying proper parameter propagation with default (False) and explicit (True) values

[github-actions]

API Change Check

APIView identified API level changes in this PR and created the following API reviews

azure-ai-evaluation

[nagkumar91]

@copilot 2025-12-09 23:16:52,740 [INFO] azure-sdk-tools: The package azure-ai-evaluation needs reformat. Run black locally to reformat.

[Copilot]

@copilot 2025-12-09 23:16:52,740 [INFO] azure-sdk-tools: The package azure-ai-evaluation needs reformat. Run black locally to reformat.

Fixed in commit 363bec4. Applied black formatting to the long lines in the evaluators list by splitting them across multiple lines.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• pypi.org
  • Triggering command: /home/REDACTED/work/azure-sdk-for-python/azure-sdk-for-python/.venv/bin/pip pip install black (dns block)
• scanning-api.github.com
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)