diff --git a/.github/PULL_REQUEST_TEMPLATE/task_submission_en.md b/.github/PULL_REQUEST_TEMPLATE/task_submission_en.md index 59ecfe046..371027095 100644 --- a/.github/PULL_REQUEST_TEMPLATE/task_submission_en.md +++ b/.github/PULL_REQUEST_TEMPLATE/task_submission_en.md @@ -1,39 +1,76 @@ - - -## Description - - -- **Task**: _Enter the full task name here_ -- **Variant**: _Enter the variant number here_ -- **Technology**: _Enter technology (e.g., SEQ, OMP, TBB, STL, MPI)_ -- **Description** of your implementation and report. - _Provide a concise summary of your implementation and report here._ - ---- - -## Checklist - - -- [ ] **CI Status**: All CI jobs (build, tests, report generation) are passing on my branch in my fork -- [ ] **Task Directory & Naming**: I have created a directory named `__` -- [ ] **Full Task Definition**: I have provided the complete task description in the pull request body. -- [ ] **clang-format**: My changes pass `clang-format` locally in my fork (no formatting errors) -- [ ] **clang-tidy**: My changes pass `clang-tidy` locally in my fork (no warnings/errors) -- [ ] **Functional Tests**: All functional tests are passing locally on my machine -- [ ] **Performance Tests**: All performance tests are passing locally on my machine -- [ ] **Branch**: I am working on a branch named exactly as my task directory (e.g., `nesterov_a_vector_sum`), not on `master`. -- [ ] **Truthful Content**: I confirm that every detail provided in this pull request is accurate and truthful to the best of my knowledge. - - +PR Title (enforced by CI): +- Pattern: [TASK] -. . . . +- Notes: `[TASK]` is optional; can be any text; there must be a dot and a space after each block. +- Example (RU): 2-12. Иванов Иван Иванович. 2341-а234. Вычисление суммы элементов вектора. +- Example (EN): 2-12. Ivanov Ivan Ivanovich. 2341-a234. Vector elements sum calculation. + +PR Body (enforced by CI): +- Use the 12 sections below exactly as titled; do not include HTML comments. +- After each label line (e.g., `Assignment:`), provide non-empty text. + +Commit Messages (enforced by CI): +- Subject pattern: (|): +- Allowed types: feat, fix, perf, test, refactor, docs, build, chore +- Allowed technology: seq, omp, mpi, stl, tbb, all; or use your task folder name instead of a technology +- Subject length: ≤ 72 characters, then one blank line +- Required body sections: [What], [Why], [How], Scope: (Task/Variant/Technology/Folder), Tests:, Local runs: +- Example subject: feat(omp|nesterov_a_vector_sum): implement parallel vector sum +- Example body: + [What] + Add OMP reduction for vector sum. + + [Why] + Improve performance and parallel coverage. + + [How] + Use #pragma omp parallel for reduction(+:sum). + + Scope: + - Task: 2 + - Variant: 12 + - Technology: omp + - Folder: nesterov_a_vector_sum + + Tests: + Added unit and perf tests. + + Local runs: + make test + +Please fill in ALL sections below (no HTML comments). Use English headers as given. + +## 1. Full name and group +Name and group: + +## 2. Assignment / Topic / Task +Assignment: + +## 3. Technology / Platform used +Technology: + +## 4. Goals of the work +Goals: + +## 5. Solution description and structure +Description: + +## 6. System requirements and build instructions +Build & Run: + +## 7. Testing and verification +Testing: + +## 8. Results +Results: + +## 9. Performance analysis +Analysis: + +## 10. Conclusions and possible improvements +Conclusions: + +## 11. Limitations / known issues +Limitations: + +## 12. CI / static analysis / code style +CI & Style: diff --git a/.github/PULL_REQUEST_TEMPLATE/task_submission_ru.md b/.github/PULL_REQUEST_TEMPLATE/task_submission_ru.md index 714ddc5ef..e94af928f 100644 --- a/.github/PULL_REQUEST_TEMPLATE/task_submission_ru.md +++ b/.github/PULL_REQUEST_TEMPLATE/task_submission_ru.md @@ -1,39 +1,76 @@ - - -## Описание - - -- **Задача**: _Введите здесь полное название задачи_ -- **Вариант**: _Введите здесь номер варианта_ -- **Технология**: _Введите технологию (например, SEQ, OMP, TBB, STL, MPI)_ -- **Описание** вашей реализации и отчёта. - _Кратко опишите вашу реализацию и содержание отчёта здесь._ - ---- - -## Чек-лист - - -- [ ] **Статус CI**: Все CI-задачи (сборка, тесты, генерация отчёта) успешно проходят на моей ветке в моем форке -- [ ] **Директория и именование задачи**: Я создал директорию с именем `<фамилия>_<первая_буква_имени>_<короткое_название_задачи>` -- [ ] **Полное описание задачи**: Я предоставил полное описание задачи в теле pull request -- [ ] **clang-format**: Мои изменения успешно проходят `clang-format` локально в моем форке (нет ошибок форматирования) -- [ ] **clang-tidy**: Мои изменения успешно проходят `clang-tidy` локально в моем форке (нет предупреждений/ошибок) -- [ ] **Функциональные тесты**: Все функциональные тесты успешно проходят локально на моей машине -- [ ] **Тесты производительности**: Все тесты производительности успешно проходят локально на моей машине -- [ ] **Ветка**: Я работаю в ветке, названной точно так же, как директория моей задачи (например, `nesterov_a_vector_sum`), а не в `master` -- [ ] **Правдивое содержание**: Я подтверждаю, что все сведения, указанные в этом pull request, являются точными и достоверными - - +Формат заголовка PR (проверяется CI): +- Шаблон: [TASK] -. . . . +- Примечания: `[TASK]` — опционально; может быть любым текстом; после каждого блока — точка и пробел. +- Пример (RU): 2-12. Иванов Иван Иванович. 2341-а234. Вычисление суммы элементов вектора. +- Example (EN): 2-12. Ivanov Ivan Ivanovich. 2341-a234. Vector elements sum calculation. + +Описание PR (проверяется CI): +- Используйте 12 разделов ниже с точными заголовками; не добавляйте HTML-комментарии. +- После каждой метки (например, `Assignment:`) укажите непустой текст. + +Описания коммитов (проверяется CI): +- Паттерн заголовка: (|): +- Допустимые type: feat, fix, perf, test, refactor, docs, build, chore +- Допустимые technology: seq, omp, mpi, stl, tbb, all; либо используйте имя папки задачи вместо technology +- Длина первой строки ≤ 72 символа, затем пустая строка +- Обязательные секции тела: [What], [Why], [How], Scope: (Task/Variant/Technology/Folder), Tests:, Local runs: +- Пример subject: feat(omp|nesterov_a_vector_sum): implement parallel vector sum +- Пример тела: + [What] + Add OMP reduction for vector sum. + + [Why] + Improve performance and parallel coverage. + + [How] + Use #pragma omp parallel for reduction(+:sum). + + Scope: + - Task: 2 + - Variant: 12 + - Technology: omp + - Folder: nesterov_a_vector_sum + + Tests: + Added unit and perf tests. + + Local runs: + make test + +Заполните ВСЕ разделы ниже (без HTML-комментариев). Заголовки разделов оставьте на английском, как указано. + +## 1. Full name and group +Name and group: + +## 2. Assignment / Topic / Task +Assignment: + +## 3. Technology / Platform used +Technology: + +## 4. Goals of the work +Goals: + +## 5. Solution description and structure +Description: + +## 6. System requirements and build instructions +Build & Run: + +## 7. Testing and verification +Testing: + +## 8. Results +Results: + +## 9. Performance analysis +Analysis: + +## 10. Conclusions and possible improvements +Conclusions: + +## 11. Limitations / known issues +Limitations: + +## 12. CI / static analysis / code style +CI & Style: diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index b6f96f76c..6540eae4f 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,6 +1,76 @@ - +PR Title (enforced by CI): +- Pattern: [TASK] -. . . . +- Notes: `[TASK]` is optional; can be any text; there must be a dot and a space after each block. +- Example (RU): 2-12. Иванов Иван Иванович. 2341-а234. Вычисление суммы элементов вектора. +- Example (EN): 2-12. Ivanov Ivan Ivanovich. 2341-a234. Vector elements sum calculation. -Please go to the `Preview` tab and select the appropriate template: +PR Body (enforced by CI): +- Use the 12 sections below exactly as titled; do not include HTML comments. +- After each label line (e.g., `Assignment:`), provide non-empty text. -* [Submit Student task (English)](?expand=1&template=task_submission_en.md) -* [Submit Student task (Russian)](?expand=1&template=task_submission_ru.md) +Commit Messages (enforced by CI): +- Subject pattern: (|): +- Allowed types: feat, fix, perf, test, refactor, docs, build, chore +- Allowed technology: seq, omp, mpi, stl, tbb, all; or use your task folder name instead of a technology +- Subject length: ≤ 72 characters, then one blank line +- Required body sections: [What], [Why], [How], Scope: (Task/Variant/Technology/Folder), Tests:, Local runs: +- Example subject: feat(omp|nesterov_a_vector_sum): implement parallel vector sum +- Example body: + [What] + Add OMP reduction for vector sum. + + [Why] + Improve performance and parallel coverage. + + [How] + Use #pragma omp parallel for reduction(+:sum). + + Scope: + - Task: 2 + - Variant: 12 + - Technology: omp + - Folder: nesterov_a_vector_sum + + Tests: + Added unit and perf tests. + + Local runs: + make test + +Please fill in ALL sections below (no HTML comments). + +## 1. Full name and group +Name and group: + +## 2. Assignment / Topic / Task +Assignment: + +## 3. Technology / Platform used +Technology: + +## 4. Goals of the work +Goals: + +## 5. Solution description and structure +Description: + +## 6. System requirements and build instructions +Build & Run: + +## 7. Testing and verification +Testing: + +## 8. Results +Results: + +## 9. Performance analysis +Analysis: + +## 10. Conclusions and possible improvements +Conclusions: + +## 11. Limitations / known issues +Limitations: + +## 12. CI / static analysis / code style +CI & Style: diff --git a/.github/scripts/validate_pr.py b/.github/scripts/validate_pr.py new file mode 100644 index 000000000..d8ef2117f --- /dev/null +++ b/.github/scripts/validate_pr.py @@ -0,0 +1,505 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +""" +PR/commit compliance validator for parallel_programming_course. + +Runs locally and in GitHub Actions. Uses only stdlib. +""" + +from __future__ import annotations + +import argparse +import json +import os +import re +import sys +from typing import List, Dict, Tuple, Optional +from urllib.request import Request, urlopen +from urllib.parse import quote + + +# --- Title validation regex (strict) --- +TITLE_REGEX = r""" +^(?:\[TASK\]\s*)? +(?P\d+)-(?P\d+)\.\s+ +(?P[А-ЯA-ZЁ][а-яa-zё]+)\s+ +(?P[А-ЯA-ZЁ][а-яa-zё]+)\s+ +(?P[А-ЯA-ZЁ][а-яa-zё]+)\.\s+ +(?P.+?)\.\s+ +(?P\S.*) +$ +""" + + +SUBJECT_REGEX = r"^(feat|fix|perf|test|refactor|docs|build|chore)\(([a-z]+)\|([a-z0-9_]+)\): [A-Za-z0-9].*$" +ALLOWED_TECH = {"seq", "omp", "mpi", "stl", "tbb", "all"} + + +def print_section(title: str) -> None: + line = "=" * 8 + print(f"\n{line} {title} {line}") + + +def _trim(s: Optional[str]) -> str: + return (s or "").strip() + + +def validate_title(title: str) -> List[str]: + """Validate PR title against strict course rules. + + Returns a list of error messages (empty if valid). + """ + errors: List[str] = [] + title = title.strip() + + example_ru = ( + "2-12. Иванов Иван Иванович. 2341-а234. Вычисление суммы элементов вектора." + ) + example_en = ( + "2-12. Ivanov Ivan Ivanovich. 2341-a234. Vector elements sum calculation." + ) + + if not title: + return [ + "PR title is empty. Expected format is strictly defined.\n" + f"Example (RU): {example_ru}\n" + f"Example (EN): {example_en}" + ] + + # Full strict match (record but do not early-return to allow granular checks) + full_match = re.match(TITLE_REGEX, title, flags=re.UNICODE | re.VERBOSE) is not None + + # If not matched – try to pinpoint the failing segment. + # 1) Optional prefix is allowed; strip it for partial checks + work = title + if work.startswith("[TASK]"): + task_prefix_len = len("[TASK]") + work = work[task_prefix_len:].lstrip() + + # 2) Task/variant with dot + m = re.match(r"^(\d+)-(\d+)\.\s+", work) + if not m: + errors.append( + "Invalid task/variant block — expected '-. ' (a dot and a space after the digits).\n" + f"Example (RU): {example_ru}\n" + f"Example (EN): {example_en}" + ) + return errors + + pos = m.end() + rest = work[pos:] + + # 3) Full name with dot after patronymic + m = re.match( + r"^([А-ЯA-ZЁ][а-яa-zё]+)\s+([А-ЯA-ZЁ][а-яa-zё]+)\s+([А-ЯA-ZЁ][а-яa-zё]+)\.\s+", + rest, + flags=re.UNICODE, + ) + if not m: + errors.append( + "Invalid full name block — expected 'LastName FirstName MiddleName.' (a dot after patronymic).\n" + f"Example (RU): {example_ru}\n" + f"Example (EN): {example_en}" + ) + return errors + + pos = m.end() + rest = rest[pos:] + + # 4) Group with dot + m = re.match(r"^(.+?)\.\s+", rest, flags=re.UNICODE) + if not m: + errors.append( + "Invalid group block — expected '. ' (a dot and a space after group).\n" + f"Example (RU): {example_ru}\n" + f"Example (EN): {example_en}" + ) + return errors + + pos = m.end() + rest = rest[pos:] + + # 5) Task name validity is enforced by the full regex (non-whitespace start) + + # If no specific segment errors and full regex matched, accept + if full_match: + return errors + # Otherwise, fallback message + errors.append( + "PR title does not match the required pattern.\n" + f"Current title: '{title}'\n" + f"Example (RU): {example_ru}\n" + f"Example (EN): {example_en}" + ) + return errors + + +def _split_sections_by_headers(body: str) -> Dict[str, Tuple[int, int]]: + """Return mapping from exact header to (start_index, end_index) slice in body. + + We look for exact level-2 headers beginning with '## ' and use their textual content + to delimit sections. + """ + headers_regex = re.compile(r"^##\s+\d+\..*$", flags=re.MULTILINE) + matches = list(headers_regex.finditer(body)) + sections: Dict[str, Tuple[int, int]] = {} + for i, m in enumerate(matches): + start = m.start() + end = matches[i + 1].start() if i + 1 < len(matches) else len(body) + next_newline = body.find("\n", m.start()) + if next_newline == -1: + next_newline = end + header_line = body[m.start() : next_newline] + sections[header_line.strip()] = (start, end) + return sections + + +BodyReq = Tuple[str, str] # (header, label) + + +REQUIRED_BODY: List[BodyReq] = [ + ("## 1. Full name and group", "Name and group:"), + ("## 2. Assignment / Topic / Task", "Assignment:"), + ("## 3. Technology / Platform used", "Technology:"), + ("## 4. Goals of the work", "Goals:"), + ("## 5. Solution description and structure", "Description:"), + ("## 6. System requirements and build instructions", "Build & Run:"), + ("## 7. Testing and verification", "Testing:"), + ("## 8. Results", "Results:"), + ("## 9. Performance analysis", "Analysis:"), + ("## 10. Conclusions and possible improvements", "Conclusions:"), + ("## 11. Limitations / known issues", "Limitations:"), + ("## 12. CI / static analysis / code style", "CI & Style:"), +] + + +def _label_value_in_section(section_text: str, label: str) -> Optional[str]: + # Look for a line that starts with the label and has any non-whitespace after colon + pattern = re.compile(rf"^{re.escape(label)}\s*(.+)$", flags=re.MULTILINE) + m = pattern.search(section_text) + if not m: + return None + value = m.group(1).strip() + return value if value else None + + +def validate_body(body: str) -> List[str]: + errors: List[str] = [] + if not body or not body.strip(): + errors.append("PR body is empty. Fill all 12 required sections.") + return errors + + if "'. Remove all guidance comments." + ) + + sections_map = _split_sections_by_headers(body) + + missing_headers: List[str] = [] + empty_labels: List[str] = [] + + for header, label in REQUIRED_BODY: + if header not in sections_map: + missing_headers.append(header) + continue + start, end = sections_map[header] + section_text = body[start:end] + value = _label_value_in_section(section_text, label) + if not value: + empty_labels.append(f"{header} → '{label}'") + + if missing_headers: + errors.append("Missing required sections:") + for h in missing_headers: + errors.append(f"✗ {h}") + + if empty_labels: + errors.append("Empty required fields (add text after the colon):") + for label_entry in empty_labels: + errors.append(f"✗ {label_entry}") + + return errors + + +def fetch_pr_commits(owner: str, repo: str, pr_number: int, token: str) -> List[Dict]: + url = f"https://api.github.com/repos/{quote(owner)}/{quote(repo)}/pulls/{pr_number}/commits" + headers = { + "Accept": "application/vnd.github+json", + "Authorization": f"Bearer {token}", + "X-GitHub-Api-Version": "2022-11-28", + "User-Agent": "parallel-programming-course-pr-validator", + } + req = Request(url, headers=headers, method="GET") + with urlopen(req) as resp: + data = json.loads(resp.read().decode("utf-8")) + return data # list of commits + + +def _extract_scope_block(body: str) -> Optional[str]: + # Find 'Scope:' line and return until next blank line or end + m = re.search(r"^Scope:\s*$", body, flags=re.MULTILINE) + if not m: + return None + start = m.end() + # From next line onward + rest = body[start:] + # Stop at first double newline + split = re.split(r"\n\s*\n", rest, maxsplit=1) + return split[0] + + +def validate_commit_message(message: str) -> List[str]: + errors: List[str] = [] + lines = message.splitlines() + subject = lines[0] if lines else "" + + # Subject line length + if len(subject) > 72: + errors.append("Subject exceeds 72 characters.") + + # Subject regex + m = re.match(SUBJECT_REGEX, subject) + if not m: + errors.append( + "Invalid subject. Expected '(|): '.\n" + "Example (EN): feat(omp|nesterov_a_vector_sum): implement parallel vector sum\n" + "Example (RU): feat(omp|nesterov_a_vector_sum): implement параллельную сумму вектора" + ) + else: + tech = m.group(2) + if tech not in ALLOWED_TECH: + errors.append( + f"Disallowed technology '{tech}'. Allowed: {', '.join(sorted(ALLOWED_TECH))}." + ) + + # Require blank line after subject and non-empty body + if len(lines) < 2 or lines[1].strip() != "": + errors.append("A blank line must follow the subject.") + + body = "\n".join(lines[2:]) if len(lines) >= 2 else "" + + # Body tokens at start of line + required_tokens = [ + r"^\[What\]", + r"^\[Why\]", + r"^\[How\]", + r"^Scope:", + r"^Tests:", + r"^Local runs:", + ] + for tok in required_tokens: + if not re.search(tok, body, flags=re.MULTILINE): + errors.append(f"Missing required body section: '{tok.strip('^')}'.") + + # Scope block must include Task, Variant, Technology, Folder + scope_block = _extract_scope_block(body) + if scope_block is None: + # already flagged by required token; keep specific scope details only if present + pass + else: + required_scope = ["Task", "Variant", "Technology", "Folder"] + for key in required_scope: + if not re.search( + rf"^\s*[-*]?\s*{key}\s*:\s*.+$", scope_block, flags=re.MULTILINE + ): + errors.append(f"In 'Scope:' section missing or empty field '{key}:'.") + + return errors + + +def _load_event_payload(path: Optional[str]) -> Optional[dict]: + if not path or not os.path.exists(path): + return None + with open(path, "r", encoding="utf-8") as f: + try: + return json.load(f) + except Exception: + return None + + +def main() -> int: + parser = argparse.ArgumentParser(description="PR/commit compliance validator") + parser.add_argument( + "--repo", + type=str, + default=os.environ.get("GITHUB_REPOSITORY"), + help="owner/repo", + ) + parser.add_argument("--pr", type=int, default=None, help="PR number") + parser.add_argument( + "--checks", type=str, choices=["title", "body", "commits", "all"], default="all" + ) + parser.add_argument("--fail-on-warn", action="store_true") + parser.add_argument("--verbose", action="store_true") + + args = parser.parse_args() + + payload = _load_event_payload(os.environ.get("GITHUB_EVENT_PATH")) + + owner: Optional[str] = None + repo: Optional[str] = None + pr_number: Optional[int] = args.pr + + if args.repo and "/" in args.repo: + owner, repo = args.repo.split("/", 1) + + if payload and not pr_number: + pr_number = payload.get("number") or ( + payload.get("pull_request", {}).get("number") + if payload.get("pull_request") + else None + ) + + # Collect title/body from payload when available + pr_title = None + pr_body = None + if payload and payload.get("pull_request"): + pr = payload["pull_request"] + pr_title = pr.get("title") + pr_body = pr.get("body") or "" + + if args.verbose: + print_section("CONFIG") + print(f"repo: {args.repo}") + print(f"owner: {owner}, repo: {repo}, pr: {pr_number}") + print(f"event payload: {'loaded' if payload else 'none'}") + + total_errors: List[str] = [] + + try: + if args.checks in ("title", "all"): + print_section("PR TITLE") + if pr_title is None: + print( + "Could not get PR title from event payload. Ensure a pull_request context or supply it manually." + ) + total_errors.append("No title data") + else: + errs = validate_title(pr_title) + if errs: + for e in errs: + if not e.startswith("✗"): + print(f"✗ {e}") + else: + print(e) + total_errors.append(e) + else: + print("OK: PR title is valid.") + + if args.checks in ("body", "all"): + print_section("PR BODY") + if pr_body is None: + print( + "Could not get PR body from event payload. Ensure a pull_request context or supply it manually." + ) + total_errors.append("No body data") + else: + errs = validate_body(pr_body) + if errs: + for e in errs: + if not e.startswith("✗"): + print(f"✗ {e}") + else: + print(e) + total_errors.append(e) + else: + print("OK: PR body is valid.") + + if args.checks in ("commits", "all"): + print_section("COMMITS") + if not (owner and repo and pr_number): + print( + "Commit validation requires --repo owner/repo and --pr or a GitHub event payload." + ) + total_errors.append("Insufficient params for commits fetch") + else: + token = os.environ.get("GITHUB_TOKEN") + if not token: + print("GITHUB_TOKEN not found in environment for API access.") + total_errors.append("No GITHUB_TOKEN") + else: + commits = fetch_pr_commits(owner, repo, int(pr_number), token) + commit_errors = 0 + for c in commits: + sha = c.get("sha", "")[:7] + message = c.get("commit", {}).get("message", "") + first = message.splitlines()[0] if message else "" + errs = validate_commit_message(message) + header = f"{sha} — {first}" + if errs: + commit_errors += 1 + print(f"Commit {header}") + for e in errs: + print(f"✗ {e}") + total_errors.extend(errs) + else: + print(f"OK: {header}") + + print_section("SUMMARY") + if total_errors: + print(f"Found {len(total_errors)} error(s)") + return 1 + else: + print("All checks passed") + return 0 + + except SystemExit: + raise + except Exception as e: + print_section("INTERNAL ERROR") + print(f"Internal error occurred: {e}", file=sys.stderr) + return 2 + + +if __name__ == "__main__": + # Lightweight self-checks printed only with --verbose when run without payload + exit_code = main() + # Do simple doctest-like probes when verbose flag present (cannot easily detect here), + # so rely on environment toggle to keep runtime minimal in CI. + if "--verbose" in sys.argv: + print_section("SELF-TESTS") + # Title positives + good_titles = [ + "[TASK] 2-12. Иванов Иван Иванович. 2341-а234. Вычисление суммы элементов вектора.", + "2-12. Иванов Иван Иванович. 2341-а234. Вычисление суммы элементов вектора.", + ] + bad_titles = [ + "Иванов Иван Иванович — задача", + "2-12 Иванов Иван Иванович. 2341-а234. …", + "2-12. Иванов Иван. 2341-а234. …", + ] + for t in good_titles: + assert not validate_title(t), f"Expected valid title: {t}" + for t in bad_titles: + assert validate_title(t), f"Expected invalid title: {t}" + + # Commit subjects + assert re.match( + SUBJECT_REGEX, + "feat(omp|nesterov_a_vector_sum): implement parallel vector sum", + ) + assert not re.match(SUBJECT_REGEX, "feature(omp|x): bad type") + # Technology validation is performed outside the regex + errs = validate_commit_message( + ( + "feat(cuda|nesterov_a_vector_sum): add cuda impl" + "\n\n[What]\n[Why]\n[How]\nScope:\n" + "- Task: 1\n- Variant: 2\n- Technology: cuda\n- Folder: nesterov_a_vector_sum\n" + "Tests:\nLocal runs:\n" + ) + ) + assert any("Disallowed technology" in e for e in errs) + too_long = "feat(omp|nesterov_a_vector_sum): " + "x" * 73 + errs = validate_commit_message( + ( + too_long + "\n\n[What]\n[Why]\n[How]\nScope:\n" + "- Task: 1\n- Variant: 2\n- Technology: omp\n- Folder: nesterov_a_vector_sum\n" + "Tests:\nLocal runs:\n" + ) + ) + assert any("exceeds 72" in e for e in errs) + print("Self-tests passed") + + sys.exit(exit_code) diff --git a/.github/workflows/main.yml b/.github/workflows/main.yml index b41003750..5c783f055 100644 --- a/.github/workflows/main.yml +++ b/.github/workflows/main.yml @@ -2,7 +2,10 @@ name: Build application on: push: + branches: + - master pull_request: + types: [opened, edited, synchronize, reopened] merge_group: schedule: - cron: '0 0 * * *' @@ -16,19 +19,20 @@ concurrency: !startsWith(github.ref, 'refs/heads/gh-readonly-queue') }} jobs: - pre-commit: - uses: ./.github/workflows/pre-commit.yml + pr_compliance: + uses: ./.github/workflows/pr-compliance.yml + ubuntu: needs: - - pre-commit + - pr_compliance uses: ./.github/workflows/ubuntu.yml mac: needs: - - pre-commit + - pr_compliance uses: ./.github/workflows/mac.yml windows: needs: - - pre-commit + - pr_compliance uses: ./.github/workflows/windows.yml perf: needs: diff --git a/.github/workflows/pr-compliance.yml b/.github/workflows/pr-compliance.yml new file mode 100644 index 000000000..4d9f94825 --- /dev/null +++ b/.github/workflows/pr-compliance.yml @@ -0,0 +1,84 @@ +name: PR Compliance + +on: + workflow_call: + +jobs: + unit_tests: + name: Unit Tests + runs-on: ubuntu-latest + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Install coverage + run: | + python -m pip install --upgrade pip + python -m pip install coverage + + - name: Run validator unit tests with coverage + run: | + coverage run -m unittest -v tests/test_validate_pr.py tests/test_validate_pr_main.py + coverage report --fail-under=100 -m .github/scripts/validate_pr.py + + pr_title: + name: Validate PR Title + runs-on: ubuntu-latest + needs: [unit_tests] + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Validate PR title + run: | + python .github/scripts/validate_pr.py --checks title --verbose + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + pr_body: + name: Validate PR Body + runs-on: ubuntu-latest + needs: [unit_tests] + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Validate PR body + run: | + python .github/scripts/validate_pr.py --checks body --verbose + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} + + pr_commits: + name: Validate PR Commits + runs-on: ubuntu-latest + needs: [unit_tests] + steps: + - name: Checkout + uses: actions/checkout@v4 + + - name: Set up Python + uses: actions/setup-python@v5 + with: + python-version: '3.11' + + - name: Validate PR commits + run: | + python .github/scripts/validate_pr.py --checks commits --verbose + env: + GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }} diff --git a/setup.cfg b/setup.cfg index cf53ebb5f..44ce9caf5 100644 --- a/setup.cfg +++ b/setup.cfg @@ -1,5 +1,6 @@ [flake8] max-line-length = 120 +extend-ignore = E203 exclude = 3rdparty venv diff --git a/tests/test_validate_pr.py b/tests/test_validate_pr.py new file mode 100644 index 000000000..c615ba6fb --- /dev/null +++ b/tests/test_validate_pr.py @@ -0,0 +1,258 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +import os +import unittest + + +ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) +SCRIPT_PATH = os.path.join(ROOT, ".github", "scripts", "validate_pr.py") + + +def _import_validator(): + import importlib.util + + spec = importlib.util.spec_from_file_location("validate_pr", SCRIPT_PATH) + mod = importlib.util.module_from_spec(spec) + assert spec.loader is not None + spec.loader.exec_module(mod) + return mod + + +class TestPRTitle(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.v = _import_validator() + + def test_title_valid_ru_and_en(self): + ok_ru = ( + "2-12. Иванов Иван Иванович. 2341-а234. Вычисление суммы элементов вектора." + ) + ok_task_tag_ru = "[TASK] " + ok_ru + ok_en = "3-7. Smith John Edward. 1234-a1. Fast matrix multiplication." + for t in (ok_ru, ok_task_tag_ru, ok_en): + with self.subTest(t=t): + self.assertEqual(self.v.validate_title(t), []) + + def test_title_invalid_examples(self): + bad = [ + "Иванов Иван Иванович — задача", # no numbers/format + "2-12 Иванов Иван Иванович. 2341-а234. …", # missing dot after 2-12 + "2-12. Иванов Иван. 2341-а234. …", # no patronymic + ] + for t in bad: + with self.subTest(t=t): + self.assertNotEqual(self.v.validate_title(t), []) + + def test_title_empty_and_task_prefix_partial(self): + # Empty title path + errs = self.v.validate_title("") + self.assertNotEqual(errs, []) + # Strip [TASK] and fail early on task/variant block to cover branch + errs = self.v.validate_title("[TASK] Иванов Иван Иванович — задача") + self.assertTrue(any("Invalid task/variant block" in e for e in errs)) + + +class TestPRBody(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.v = _import_validator() + + def test_body_valid(self): + body = ( + "## 1. Full name and group\n" + "Name and group: Ivanov Ivan, 2341-a234\n\n" + "## 2. Assignment / Topic / Task\n" + "Assignment: Implement vector sum\n\n" + "## 3. Technology / Platform used\n" + "Technology: omp\n\n" + "## 4. Goals of the work\n" + "Goals: Learn parallel reduction\n\n" + "## 5. Solution description and structure\n" + "Description: Parallelize with OpenMP reduction\n\n" + "## 6. System requirements and build instructions\n" + "Build & Run: cmake ..; make; ./app\n\n" + "## 7. Testing and verification\n" + "Testing: Unit and perf tests\n\n" + "## 8. Results\n" + "Results: Speedup 3x\n\n" + "## 9. Performance analysis\n" + "Analysis: Scaling on 8 threads\n\n" + "## 10. Conclusions and possible improvements\n" + "Conclusions: Improve memory layout\n\n" + "## 11. Limitations / known issues\n" + "Limitations: Small N overhead\n\n" + "## 12. CI / static analysis / code style\n" + "CI & Style: Lint, formatting, CI passing\n" + ) + self.assertEqual(self.v.validate_body(body), []) + + def test_body_missing_header_and_empty_label(self): + body = ( + "## 1. Full name and group\n" + "Name and group: \n\n" # empty label + # Missing header 2 entirely + "## 3. Technology / Platform used\n" + "Technology: omp\n\n" + "## 4. Goals of the work\n" + "Goals: x\n\n" + "## 5. Solution description and structure\n" + "Description: x\n\n" + "## 6. System requirements and build instructions\n" + "Build & Run: x\n\n" + "## 7. Testing and verification\n" + "Testing: x\n\n" + "## 8. Results\n" + "Results: x\n\n" + "## 9. Performance analysis\n" + "Analysis: x\n\n" + "## 10. Conclusions and possible improvements\n" + "Conclusions: x\n\n" + "## 11. Limitations / known issues\n" + "Limitations: x\n\n" + "## 12. CI / static analysis / code style\n" + "CI & Style: x\n" + ) + errs = self.v.validate_body(body) + self.assertTrue(any("Missing required sections:" in e for e in errs)) + self.assertTrue(any("Empty required fields" in e for e in errs)) + + def test_body_with_html_comments(self): + body = ( + "## 1. Full name and group\n" + "Name and group: x\n\n" + "\n" + "## 2. Assignment / Topic / Task\n" + "Assignment: x\n\n" + "## 3. Technology / Platform used\n" + "Technology: x\n\n" + "## 4. Goals of the work\n" + "Goals: x\n\n" + "## 5. Solution description and structure\n" + "Description: x\n\n" + "## 6. System requirements and build instructions\n" + "Build & Run: x\n\n" + "## 7. Testing and verification\n" + "Testing: x\n\n" + "## 8. Results\n" + "Results: x\n\n" + "## 9. Performance analysis\n" + "Analysis: x\n\n" + "## 10. Conclusions and possible improvements\n" + "Conclusions: x\n\n" + "## 11. Limitations / known issues\n" + "Limitations: x\n\n" + "## 12. CI / static analysis / code style\n" + "CI & Style: x\n" + ) + errs = self.v.validate_body(body) + self.assertTrue(any("Found HTML comments" in e for e in errs)) + + def test_body_empty(self): + errs = self.v.validate_body("") + self.assertTrue(any("PR body is empty" in e for e in errs)) + + def test_body_label_without_any_value(self): + body = ( + "## 1. Full name and group\n" + "Name and group:\n\n" # no spaces after colon + "## 2. Assignment / Topic / Task\n" + "Assignment: A\n\n" + "## 3. Technology / Platform used\n" + "Technology: A\n\n" + "## 4. Goals of the work\n" + "Goals: A\n\n" + "## 5. Solution description and structure\n" + "Description: A\n\n" + "## 6. System requirements and build instructions\n" + "Build & Run: A\n\n" + "## 7. Testing and verification\n" + "Testing: A\n\n" + "## 8. Results\n" + "Results: A\n\n" + "## 9. Performance analysis\n" + "Analysis: A\n\n" + "## 10. Conclusions and possible improvements\n" + "Conclusions: A\n\n" + "## 11. Limitations / known issues\n" + "Limitations: A\n\n" + "## 12. CI / static analysis / code style\n" + "CI & Style: A\n" + ) + errs = self.v.validate_body(body) + self.assertTrue(any("Empty required fields" in e for e in errs)) + + +class TestCommitMessages(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.v = _import_validator() + + def test_commit_valid(self): + msg = ( + "feat(omp|nesterov_a_vector_sum): implement parallel vector sum\n\n" + "[What]\nAdd OMP reduction for vector sum.\n\n" + "[Why]\nImprove performance.\n\n" + "[How]\nUse #pragma omp parallel for reduction(+:sum).\n\n" + "Scope:\n- Task: 2\n- Variant: 12\n- Technology: omp\n- Folder: nesterov_a_vector_sum\n\n" + "Tests:\nAdded unit and perf tests.\n\n" + "Local runs:\nmake test\n" + ) + self.assertEqual(self.v.validate_commit_message(msg), []) + + def test_commit_invalid_cases(self): + # wrong type + msg1 = ( + "feature(omp|nesterov_a_vector_sum): summary\n\n[What]\n[Why]\n[How]\n" + "Scope:\n- Task: 1\n- Variant: 1\n- Technology: omp\n- Folder: f\n" + "Tests:\nLocal runs:\n" + ) + # disallowed technology + msg2 = ( + "feat(cuda|nesterov_a_vector_sum): add cuda impl\n\n[What]\n[Why]\n[How]\n" + "Scope:\n- Task: 1\n- Variant: 1\n- Technology: cuda\n- Folder: f\n" + "Tests:\nLocal runs:\n" + ) + # subject too long + long_summary = "x" * 73 + msg3 = ( + f"feat(omp|nesterov_a_vector_sum): {long_summary}\n\n[What]\n[Why]\n[How]\n" + "Scope:\n- Task: 1\n- Variant: 1\n- Technology: omp\n- Folder: f\n" + "Tests:\nLocal runs:\n" + ) + # no blank line + msg4 = ( + "feat(omp|nesterov_a_vector_sum): ok\n[What]\n[Why]\n[How]\n" + "Scope:\n- Task: 1\n- Variant: 1\n- Technology: omp\n- Folder: f\n" + "Tests:\nLocal runs:\n" + ) + # missing tokens + msg5 = "feat(omp|nesterov_a_vector_sum): ok\n\nNo sections here\n" + # missing fields in scope + msg6 = ( + "feat(omp|nesterov_a_vector_sum): ok\n\n[What]\n[Why]\n[How]\n" + "Scope:\n- Task: 1\n- Technology: omp\n- Folder: f\n\n" + "Tests:\nLocal runs:\n" + ) + + for i, m in enumerate([msg1, msg2, msg3, msg4, msg5, msg6], start=1): + with self.subTest(i=i): + errs = self.v.validate_commit_message(m) + self.assertNotEqual(errs, []) + + def test_title_partial_group_and_taskname_errors_and_fallback(self): + # Arbitrary group should be accepted + t1 = "2-12. Иванов Иван Иванович. ХХХ. Что-то" + self.assertEqual(self.v.validate_title(t1), []) + # Trailing newline after group triggers fallback (missing taskname) + t2 = "2-12. Иванов Иван Иванович. 2341-а234. \n" + errs = self.v.validate_title(t2) + self.assertNotEqual(errs, []) + # Fallback case: embed newline so full regex fails while partial checks pass + t3 = "2-12. Иванов Иван Иванович. 2341-а234. Задача\nещё" + errs = self.v.validate_title(t3) + self.assertTrue(any("does not match the required pattern" in e for e in errs)) + + +if __name__ == "__main__": + unittest.main(verbosity=2) diff --git a/tests/test_validate_pr_main.py b/tests/test_validate_pr_main.py new file mode 100644 index 000000000..b278833e3 --- /dev/null +++ b/tests/test_validate_pr_main.py @@ -0,0 +1,310 @@ +#!/usr/bin/env python3 +# -*- coding: utf-8 -*- + +import json +import os +import runpy +import sys +import tempfile +import unittest +from unittest import mock + + +ROOT = os.path.abspath(os.path.join(os.path.dirname(__file__), "..")) +SCRIPT_PATH = os.path.join(ROOT, ".github", "scripts", "validate_pr.py") + + +def _import_validator(): + import importlib.util + + spec = importlib.util.spec_from_file_location("validate_pr", SCRIPT_PATH) + mod = importlib.util.module_from_spec(spec) + assert spec.loader is not None + spec.loader.exec_module(mod) + # Ensure it's importable by name for mocks if needed + import sys as _sys + + _sys.modules.setdefault("validate_pr", mod) + return mod + + +class TestHelpersAndPayload(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.v = _import_validator() + + def test_trim_helper(self): + self.assertEqual(self.v._trim(None), "") + self.assertEqual(self.v._trim(" a "), "a") + + def test_load_event_payload(self): + # Non-existing path + self.assertIsNone(self.v._load_event_payload("/nonexistent/path.json")) + # Invalid JSON + with tempfile.NamedTemporaryFile("w+", delete=False) as tf: + tf.write("not json") + path_bad = tf.name + try: + self.assertIsNone(self.v._load_event_payload(path_bad)) + finally: + os.unlink(path_bad) + # Valid JSON + data = {"pull_request": {"number": 1, "title": "x", "body": "y"}} + with tempfile.NamedTemporaryFile("w+", delete=False) as tf: + json.dump(data, tf) + path_ok = tf.name + try: + loaded = self.v._load_event_payload(path_ok) + self.assertIsInstance(loaded, dict) + self.assertIn("pull_request", loaded) + finally: + os.unlink(path_ok) + + def test_split_headers_without_trailing_newline(self): + # Cover branch where a header has no trailing newline at EOF + body = "## 1. Full name and group" + sections = self.v._split_sections_by_headers(body) + self.assertIn("## 1. Full name and group", sections) + + +class DummyHTTPResponse: + def __init__(self, payload: bytes): + self._payload = payload + + def __enter__(self): + return self + + def __exit__(self, exc_type, exc, tb): + return False + + def read(self): + return self._payload + + +class TestFetchAndMain(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.v = _import_validator() + + def test_fetch_pr_commits_mocked(self): + commits = [ + { + "sha": "1234567", + "commit": { + "message": ( + "feat(omp|t): m\n\n[What]\n[Why]\n[How]\nScope:\n" + "- Task: 1\n- Variant: 1\n- Technology: omp\n- Folder: t\n" + "Tests:\nLocal runs:\n" + ) + }, + } + ] + payload = json.dumps(commits).encode("utf-8") + with mock.patch.object( + self.v, "urlopen", return_value=DummyHTTPResponse(payload) + ): + res = self.v.fetch_pr_commits("o", "r", 1, "t") + self.assertEqual(len(res), 1) + + def test_main_commits_insufficient_params(self): + # No repo/pr and no payload + with mock.patch.object(sys, "argv", ["prog", "--checks", "commits"]): + # Ensure no event path + os.environ.pop("GITHUB_EVENT_PATH", None) + code = self.v.main() + self.assertEqual(code, 1) + + def test_main_commits_missing_token(self): + # Have repo/pr but no token + with mock.patch.object( + sys, "argv", ["prog", "--checks", "commits", "--repo", "o/r", "--pr", "1"] + ): + os.environ.pop("GITHUB_TOKEN", None) + code = self.v.main() + self.assertEqual(code, 1) + + def test_main_commits_ok(self): + # Valid commit and token supplied + ok_msg = ( + "feat(omp|nesterov_a_vector_sum): ok\n\n[What]\n[Why]\n[How]\n" + "Scope:\n- Task: 1\n- Variant: 1\n- Technology: omp\n- Folder: nesterov_a_vector_sum\n\n" + "Tests:\nX\n\nLocal runs:\nX\n" + ) + commits = [{"sha": "abcdef0", "commit": {"message": ok_msg}}] + with mock.patch.object( + sys, "argv", ["prog", "--checks", "commits", "--repo", "o/r", "--pr", "1"] + ): + os.environ["GITHUB_TOKEN"] = "token" + with mock.patch.object(self.v, "fetch_pr_commits", return_value=commits): + code = self.v.main() + self.assertEqual(code, 0) + + def test_main_commits_error_and_internal(self): + bad_msg = "feat(omp|task): bad no blank line\n[What]" + commits = [{"sha": "abcdef0", "commit": {"message": bad_msg}}] + with mock.patch.object( + sys, "argv", ["prog", "--checks", "commits", "--repo", "o/r", "--pr", "1"] + ): + os.environ["GITHUB_TOKEN"] = "token" + with mock.patch.object(self.v, "fetch_pr_commits", return_value=commits): + code = self.v.main() + self.assertEqual(code, 1) + + def test_main_title_and_body_missing_in_payload(self): + # No payload at all + with mock.patch.object(sys, "argv", ["prog", "--checks", "title", "--verbose"]): + os.environ.pop("GITHUB_EVENT_PATH", None) + code = self.v.main() + self.assertEqual(code, 1) + # Payload present but missing title/body triggers error printing branch + data = { + "pull_request": { + "number": 5, + "title": "Иванов Иван Иванович — задача", + "body": "## 1. Full name and group\nName and group: \n", + } + } + with tempfile.NamedTemporaryFile("w+", delete=False) as tf: + json.dump(data, tf) + path = tf.name + try: + with mock.patch.object( + sys, "argv", ["prog", "--checks", "title", "--verbose"] + ): + os.environ["GITHUB_EVENT_PATH"] = path + code = self.v.main() + self.assertEqual(code, 1) + with mock.patch.object( + sys, "argv", ["prog", "--checks", "body", "--verbose"] + ): + os.environ["GITHUB_EVENT_PATH"] = path + code = self.v.main() + self.assertEqual(code, 1) + finally: + os.unlink(path) + + # Now raise an internal error + with mock.patch.object( + sys, "argv", ["prog", "--checks", "commits", "--repo", "o/r", "--pr", "1"] + ): + os.environ["GITHUB_TOKEN"] = "token" + with mock.patch.object( + self.v, "fetch_pr_commits", side_effect=RuntimeError("boom") + ): + code = self.v.main() + self.assertEqual(code, 2) + + def test_main_body_none_and_ok(self): + v = _import_validator() + # Body None path (no payload) + with mock.patch.object(sys, "argv", ["prog", "--checks", "body", "--verbose"]): + os.environ.pop("GITHUB_EVENT_PATH", None) + code = v.main() + self.assertEqual(code, 1) + # Body OK path + body = ( + "## 1. Full name and group\nName and group: A\n\n" + "## 2. Assignment / Topic / Task\nAssignment: A\n\n" + "## 3. Technology / Platform used\nTechnology: omp\n\n" + "## 4. Goals of the work\nGoals: A\n\n" + "## 5. Solution description and structure\nDescription: A\n\n" + "## 6. System requirements and build instructions\nBuild & Run: A\n\n" + "## 7. Testing and verification\nTesting: A\n\n" + "## 8. Results\nResults: A\n\n" + "## 9. Performance analysis\nAnalysis: A\n\n" + "## 10. Conclusions and possible improvements\nConclusions: A\n\n" + "## 11. Limitations / known issues\nLimitations: A\n\n" + "## 12. CI / static analysis / code style\nCI & Style: A\n" + ) + data = {"pull_request": {"number": 7, "title": "x", "body": body}} + with tempfile.NamedTemporaryFile("w+", delete=False) as tf: + json.dump(data, tf) + path = tf.name + try: + with mock.patch.object( + sys, "argv", ["prog", "--checks", "body", "--verbose"] + ): + os.environ["GITHUB_EVENT_PATH"] = path + code = v.main() + self.assertEqual(code, 0) + finally: + os.unlink(path) + + def test_main_title_print_branch_else(self): + v = _import_validator() + data = {"pull_request": {"number": 8, "title": "bad", "body": ""}} + with tempfile.NamedTemporaryFile("w+", delete=False) as tf: + json.dump(data, tf) + path = tf.name + try: + with mock.patch.object(sys, "argv", ["prog", "--checks", "title"]): + # Force errs starting with '✗' to hit print(e) + with mock.patch.object( + v, "validate_title", return_value=["✗ forced-error"] + ): + os.environ["GITHUB_EVENT_PATH"] = path + code = v.main() + self.assertEqual(code, 1) + finally: + os.unlink(path) + + +class TestRunAsMain(unittest.TestCase): + def test_run_module_as_main_with_verbose(self): + # Execute the script as __main__ under coverage to hit self-tests block + data = { + "number": 999, + "pull_request": { + "number": 999, + "title": "2-12. Иванов Иван Иванович. 2341-а234. Вычисление суммы элементов вектора.", + "body": ( + "## 1. Full name and group\nName and group: A\n\n" + "## 2. Assignment / Topic / Task\nAssignment: A\n\n" + "## 3. Technology / Platform used\nTechnology: omp\n\n" + "## 4. Goals of the work\nGoals: A\n\n" + "## 5. Solution description and structure\nDescription: A\n\n" + "## 6. System requirements and build instructions\nBuild & Run: A\n\n" + "## 7. Testing and verification\nTesting: A\n\n" + "## 8. Results\nResults: A\n\n" + "## 9. Performance analysis\nAnalysis: A\n\n" + "## 10. Conclusions and possible improvements\nConclusions: A\n\n" + "## 11. Limitations / known issues\nLimitations: A\n\n" + "## 12. CI / static analysis / code style\nCI & Style: A\n" + ), + }, + } + with tempfile.NamedTemporaryFile("w+", delete=False) as tf: + json.dump(data, tf) + path = tf.name + try: + os.environ["GITHUB_EVENT_PATH"] = path + # Prepare argv for verbose title check to trigger self-tests + argv = [SCRIPT_PATH, "--checks", "title", "--verbose"] + with mock.patch.object(sys, "argv", argv): + with self.assertRaises(SystemExit) as cm: + runpy.run_path(SCRIPT_PATH, run_name="__main__") + self.assertEqual(cm.exception.code, 0) + finally: + os.unlink(path) + + def test_systemexit_branch_in_try(self): + # Force validate_title to raise SystemExit inside try block to hit 'except SystemExit: raise' + v = _import_validator() + with mock.patch.object(sys, "argv", ["prog", "--checks", "title"]): + with mock.patch.object(v, "validate_title", side_effect=SystemExit(5)): + # Provide minimal payload with title + data = {"pull_request": {"number": 1, "title": "bad", "body": ""}} + with tempfile.NamedTemporaryFile("w+", delete=False) as tf: + json.dump(data, tf) + path = tf.name + try: + os.environ["GITHUB_EVENT_PATH"] = path + with self.assertRaises(SystemExit) as cm: + v.main() + self.assertEqual(cm.exception.code, 5) + finally: + os.unlink(path) + + +if __name__ == "__main__": + unittest.main(verbosity=2)