Document alignment calculation methodology for AI test case evaluation

[Copilot]

Adds detailed explanation of how individual test case alignment is calculated when evaluating AI-generated responses against human grades on a 1-5 scale.

Changes

• Extended faqs-chatbot.md evaluation section with alignment calculation methodology
• Added formula: alignment = 100% * (1 - |AI - Human| / 4)
• Added complete 5×5 alignment matrix showing all grade combinations
• Documented properties: symmetric, linear interpolation, diagonal = 100%, edges = 0%

Context

The alignment metric provides a linear interpolation from 100% (perfect match) to 0% (maximum difference), where each grade step reduces alignment by 25%. This quantifies how closely AI performance matches human expectations across test datasets.

Original prompt

Add information on how alignment is calculated for individual test cases:

"Here is a clean interpolation from 100% (perfect match) to 0% (maximally different) starting from the diagonal.

✔️ Interpretation

When AI grade = Human grade, alignment = 100%

As the grade difference increases, alignment decreases linearly

Maximum difference on a 1–5 scale is 4 steps, which maps to 0%

✔️ Formula used
alignment

100
%
×
(
1
−
∣
𝐴
𝐼
−
𝐻
𝑢
𝑚
𝑎
𝑛
∣
4
)
alignment=100%×(1−
4
∣AI−Human∣
​

)
✔️ Resulting alignment matrix
AI ↓ / Human → 1 2 3 4 5
1 100% 75% 50% 25% 0%
2 75% 100% 75% 50% 25%
3 50% 75% 100% 75% 50%
4 25% 50% 75% 100% 75%
5 0% 25% 50% 75% 100%
✔️ Properties

Symmetric

Diagonal = 100%

Far edges = 0%

Smooth, linear interpolation"

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.