-
Notifications
You must be signed in to change notification settings - Fork 33
Open
Description
cloud you tell me the result unit? like "Time To First Token", it's second or ms
=================================== Summary ====================================
Provider : openai
Model : /data/model/baichuan2-13b-chat/
Prompt Tokens : 39.0
Generation Tokens : 2048
Stream : True
Temperature : 1.0
Logprobs : None
Concurrency : QPS 50.0 constant
Time To First Token: 5.705300167132269
Latency Per Token : 135.50119360148753
Num Tokens : 258.92857142857144
Total Latency : 28838.560053018486
Num Requests : 112
Qps : 2.0004955480459414
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels