Skip to content

feat: Add timing and QUIET mode to Makefile #121

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
angerman wants to merge 2 commits into
base: stable-ghc-9.14
Choose a base branch
from
Open

feat: Add timing and QUIET mode to Makefile #121

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
2cc434f
feat(Makefile): add QUIET mode and timing for build phases
angerman Dec 8, 2025
21a35da
feat(CI): add CPU/memory metrics collection and visualization
angerman Dec 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
166 changes: 163 additions & 3 deletions .github/workflows/ci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,6 +38,45 @@ jobs:
minimal: true
ghc: true

- name: Debug R2 upload test
env:
AWS_ACCESS_KEY_ID: ${{ secrets.R2_METRICS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_METRICS_SECRET_ACCESS_KEY }}
R2_ENDPOINT: ${{ secrets.R2_METRICS_ENDPOINT }}
R2_PUBLIC_URL: ${{ vars.R2_METRICS_PUBLIC_URL }}
R2_BUCKET: ${{ vars.R2_METRICS_BUCKET_NAME }}
shell: devx {0}
run: |
echo "Testing R2 upload connectivity..."
echo "AWS_ACCESS_KEY_ID length: ${#AWS_ACCESS_KEY_ID}"
echo "AWS_SECRET_ACCESS_KEY length: ${#AWS_SECRET_ACCESS_KEY}"
echo "R2_ENDPOINT length: ${#R2_ENDPOINT}"
echo "R2_BUCKET length: ${#R2_BUCKET}"

# Hash secrets for comparison (without revealing them)
echo "Hash comparison (expected vs actual):"
echo "ACCESS_KEY_ID: 5ca6a5e1... vs $(echo "${AWS_ACCESS_KEY_ID}" | tr -d '\n' | sha256sum | cut -c1-8)..."
echo "SECRET_ACCESS_KEY: 9c22a347... vs $(echo "${AWS_SECRET_ACCESS_KEY}" | tr -d '\n' | sha256sum | cut -c1-8)..."
echo "R2_ENDPOINT: e2178c75... vs $(echo "${R2_ENDPOINT}" | tr -d '\n' | sha256sum | cut -c1-8)..."
echo "R2_BUCKET: 617649ec... vs $(echo "${R2_BUCKET}" | tr -d '\n' | sha256sum | cut -c1-8)..."

# Create test file
echo "test upload $(date)" > /tmp/r2-test.txt

# Try upload (--region auto is required for R2)
# Use AWS_ENDPOINT_URL env var, bucket from variable
export AWS_ENDPOINT_URL="${R2_ENDPOINT}"
export AWS_DEFAULT_REGION=auto
nix-shell --keep AWS_ACCESS_KEY_ID --keep AWS_SECRET_ACCESS_KEY --keep AWS_ENDPOINT_URL --keep AWS_DEFAULT_REGION -p awscli2 --run "aws s3 cp /tmp/r2-test.txt s3://${R2_BUCKET}/test/r2-test.txt"

# Check result
if [[ $? -eq 0 ]]; then
echo "SUCCESS: R2 upload works!"
echo "Public URL would be: ${R2_PUBLIC_URL}/test/r2-test.txt"
else
echo "FAILED: R2 upload did not work"
fi

- name: Update hackage
shell: devx {0}
run: cabal update
Expand All @@ -50,19 +89,130 @@ jobs:
# shell: devx {0}
# run: ./configure

- name: Start metrics collection
shell: devx {0}
run: ./mk/collect-metrics.sh start _build/metrics 0.5

- name: Build the bindist
shell: devx {0}
run: make CABAL=$PWD/_build/stage0/bin/cabal
run: make QUIET=1 CABAL=$PWD/_build/stage0/bin/cabal

- name: Upload artifacts
- name: Display build timings
if: ${{ !cancelled() }}
shell: devx {0}
run: make timing-summary

- name: Upload bindist artifacts
uses: actions/upload-artifact@v4
with:
name: ${{ matrix.plat }}-bindist
path: _build/bindist

- name: Upload build logs and timing
uses: actions/upload-artifact@v4
if: ${{ !cancelled() }} # upload logs even if the build failed
with:
name: ${{ matrix.plat }}-build-logs
path: |
_build/logs/
_build/timing/

- name: Run the testsuite
shell: devx {0}
run: make test CABAL=$PWD/_build/stage0/bin/cabal
run: make test QUIET=1 CABAL=$PWD/_build/stage0/bin/cabal

- name: Stop metrics collection
if: ${{ !cancelled() }}
shell: devx {0}
run: ./mk/collect-metrics.sh stop _build/metrics

- name: Display test timings
if: ${{ !cancelled() }}
shell: devx {0}
run: make timing-summary

- name: Generate metrics plots
if: ${{ !cancelled() }}
continue-on-error: true
shell: devx {0}
run: |
# Generate separate build and test SVG plots (wider aspect ratio)
nix-shell -p 'python3.withPackages (ps: [ps.matplotlib])' --run 'python3 ./mk/plot-metrics.py _build/metrics _build/timing _build/metrics/metrics'

- name: Upload metrics plots to R2
if: ${{ !cancelled() }}
env:
AWS_ACCESS_KEY_ID: ${{ secrets.R2_METRICS_ACCESS_KEY_ID }}
AWS_SECRET_ACCESS_KEY: ${{ secrets.R2_METRICS_SECRET_ACCESS_KEY }}
R2_ENDPOINT: ${{ secrets.R2_METRICS_ENDPOINT }}
R2_PUBLIC_URL: ${{ vars.R2_METRICS_PUBLIC_URL }}
R2_BUCKET: ${{ vars.R2_METRICS_BUCKET_NAME }}
shell: devx {0}
run: |
RUN_ID="${{ github.run_id }}"
PLAT="${{ matrix.plat }}"

export AWS_ENDPOINT_URL="${R2_ENDPOINT}"
export AWS_DEFAULT_REGION=auto

# Upload build plot if it exists
if [[ -f "_build/metrics/metrics-build.svg" ]]; then
BUILD_PATH="runs/${RUN_ID}/${PLAT}-build.svg"
nix-shell --keep AWS_ACCESS_KEY_ID --keep AWS_SECRET_ACCESS_KEY --keep AWS_ENDPOINT_URL --keep AWS_DEFAULT_REGION -p awscli2 --run "aws s3 cp _build/metrics/metrics-build.svg s3://${R2_BUCKET}/${BUILD_PATH} --content-type 'image/svg+xml'" || echo "Failed to upload build plot (non-fatal)"
echo "BUILD_PLOT_URL=${R2_PUBLIC_URL}/${BUILD_PATH}" >> $GITHUB_ENV
fi

# Upload test plot if it exists
if [[ -f "_build/metrics/metrics-test.svg" ]]; then
TEST_PATH="runs/${RUN_ID}/${PLAT}-test.svg"
nix-shell --keep AWS_ACCESS_KEY_ID --keep AWS_SECRET_ACCESS_KEY --keep AWS_ENDPOINT_URL --keep AWS_DEFAULT_REGION -p awscli2 --run "aws s3 cp _build/metrics/metrics-test.svg s3://${R2_BUCKET}/${TEST_PATH} --content-type 'image/svg+xml'" || echo "Failed to upload test plot (non-fatal)"
echo "TEST_PLOT_URL=${R2_PUBLIC_URL}/${TEST_PATH}" >> $GITHUB_ENV
fi

- name: Write metrics summary
if: ${{ !cancelled() }}
shell: devx {0}
run: |
echo "## Build Metrics Summary" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY

# Embed build plot if available
if [[ -n "${BUILD_PLOT_URL:-}" ]]; then
echo "### Build Phases (CPU & Memory)" >> $GITHUB_STEP_SUMMARY
echo "<img src=\"${BUILD_PLOT_URL}\" alt=\"Build Metrics\" width=\"100%\">" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
fi

# Embed test plot if available
if [[ -n "${TEST_PLOT_URL:-}" ]]; then
echo "### Test Phase (CPU & Memory)" >> $GITHUB_STEP_SUMMARY
echo "<img src=\"${TEST_PLOT_URL}\" alt=\"Test Metrics\" width=\"100%\">" >> $GITHUB_STEP_SUMMARY
echo "" >> $GITHUB_STEP_SUMMARY
fi

echo "### Phase Timings" >> $GITHUB_STEP_SUMMARY
echo "| Phase | Duration | Status |" >> $GITHUB_STEP_SUMMARY
echo "|-------|----------|--------|" >> $GITHUB_STEP_SUMMARY
for phase in cabal stage1 stage2 stage2-utils bindist test; do
if [[ -f "_build/timing/$phase.start" && -f "_build/timing/$phase.end" ]]; then
start=$(cat "_build/timing/$phase.start")
end=$(cat "_build/timing/$phase.end")
duration=$((end - start))
mins=$((duration / 60))
secs=$((duration % 60))
status="OK"
if [[ -f "_build/timing/$phase.status" ]]; then
[[ $(cat "_build/timing/$phase.status") == "1" ]] && status="FAIL"
fi
echo "| $phase | ${mins}m ${secs}s | $status |" >> $GITHUB_STEP_SUMMARY
fi
done
echo "" >> $GITHUB_STEP_SUMMARY
# Add memory stats from metrics
if [[ -f "_build/metrics/metrics.csv" ]]; then
max_mem=$(awk -F',' 'NR>1 {if($3>max) max=$3} END {printf "%.1f", max/1024}' _build/metrics/metrics.csv)
echo "**Peak Memory:** ${max_mem} GB" >> $GITHUB_STEP_SUMMARY
fi

- name: Upload test results
uses: actions/upload-artifact@v4
Expand All @@ -73,3 +223,13 @@ jobs:
_build/test-perf.csv
_build/test-summary.txt
_build/test-junit.xml
_build/logs/test.log
_build/timing/

- name: Upload metrics
uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: ${{ matrix.plat }}-metrics
path: |
_build/metrics/
Loading
Loading