Skip to content

feat: support multiple completions#26

Open
BaiMoHan wants to merge 4 commits intofw-ai:mainfrom
BaiMoHan:main
Open

feat: support multiple completions#26
BaiMoHan wants to merge 4 commits intofw-ai:mainfrom
BaiMoHan:main

Conversation

@BaiMoHan
Copy link

This PR adds support for multiple completions (n > 1) and improves the QPS handling mechanism. The changes include:

  1. Support for multiple completions:

    • Remove assertion that limited choices to single completion
    • Add support for handling multiple completions when n > 1
    • Concatenate multiple completion texts with newlines
    • Use logprobs from first completion for metrics
  2. Fix token count validation:

    • Fix AttributeError by using self.environment.parsed_options
    • Update token count validation to properly handle n > 1 cases
    • Improve warning message to show max_tokens and n values

Testing:

  • Tested with n=4 completions
  • Verified token count validation

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant

Comments