Skip to content

Reduce frequency of alerts #57

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
dbtavernerffdc merged 10 commits into from
Dec 17, 2025
Merged

Reduce frequency of alerts #57

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
f9fff85
Filter out recent ui alerts for a specific tests and relevant tests
moladimejicertinia Dec 15, 2025
34a6923
fix failing test
moladimejicertinia Dec 15, 2025
73fef0f
Revert change to alertType
sbrocklehurstcertinia Dec 16, 2025
cc6a7b4
Revert removal of constant
sbrocklehurstcertinia Dec 16, 2025
022f50b
Update benchmarker version
sbrocklehurstcertinia Dec 16, 2025
e53c945
Update CHANGELOG
sbrocklehurstcertinia Dec 16, 2025
b74bbd8
Revert removal of assertions
sbrocklehurstcertinia Dec 16, 2025
947e084
Update benchmarker version
sbrocklehurstcertinia Dec 16, 2025
649a08b
Revert formatting
sbrocklehurstcertinia Dec 16, 2025
fdad062
Revert removal of import
sbrocklehurstcertinia Dec 16, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# Benchmarker - Changelog

## 6.1.0

- UI performance alerts are now suppressed if the same test suite and individual test have already triggered an alert within the last 3 days.

## 6.0.2

- Update logic to generate UI Alerts to handle averages that are decimals.
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@apexdevtools/benchmarker",
"version": "6.0.2",
"version": "6.1.0",
"description": "Benchmarks performance of processes on Salesforce Orgs",
"author": {
"name": "Apex Dev Tools Team",
Expand Down
34 changes: 34 additions & 0 deletions src/database/uiAlertInfo.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -128,3 +128,37 @@ export async function getAverageLimitValuesFromDB(
return {};
}
}

export async function checkRecentUiAlerts(
suiteAndTestNamePairs: { testSuiteName: string; individualTestName: string }[]
) {
const connection = await getConnection();

const suiteAndTestNameConditions = suiteAndTestNamePairs
.map(pair => `('${pair.testSuiteName}', '${pair.individualTestName}')`)
.join(', ');

const query = `
SELECT test_suite_name, individual_test_name
FROM performance.ui_alert
WHERE create_date_time >= CURRENT_DATE - INTERVAL '3 days'
AND (test_suite_name, individual_test_name) IN (${suiteAndTestNameConditions})
`;

const existingAlerts = new Set<string>();

try {
const result = await connection.query(query);
result.forEach(
(row: { test_suite_name: string; individual_test_name: string }) => {
existingAlerts.add(
`${row.test_suite_name}_${row.individual_test_name}`
);
}
);
} catch (error) {
console.error('Error checking recent UI alerts:', error);
}

return existingAlerts;
}
18 changes: 16 additions & 2 deletions src/services/result/uiAlert.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,10 @@ import {
getNormalComponentLoadThreshold,
getCriticalComponentLoadThreshold,
} from '../../shared/env';
import { getAverageLimitValuesFromDB } from '../../database/uiAlertInfo';
import {
getAverageLimitValuesFromDB,
checkRecentUiAlerts,
} from '../../database/uiAlertInfo';
import { UiAlert } from '../../database/entity/uiAlert';
import { UiTestResultDTO } from '../../database/uiTestResult';
import { NORMAL, CRITICAL } from '../../shared/constants';
Expand All @@ -31,14 +34,25 @@ export async function generateValidAlerts(
individualTestName: result.individualTestName,
}));

const existingAlerts = await checkRecentUiAlerts(suiteAndTestNamePairs);

const alertsToProcess = needToStoreAlert.filter(
item =>
!existingAlerts.has(`${item.testSuiteName}_${item.individualTestName}`)
);

if (alertsToProcess.length === 0) {
return [];
}

// Fetch average values
const preFetchedAverages = await getAverageLimitValuesFromDB(
suiteAndTestNamePairs
);

// Generate alerts
const alerts = await Promise.all(
needToStoreAlert.map(async item =>
alertsToProcess.map(async item =>
addAlertByComparingAvg(item, preFetchedAverages)
)
);
Expand Down
68 changes: 68 additions & 0 deletions test/database/uiAlertInfo.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
import {
getAverageLimitValuesFromDB,
saveAlerts,
checkRecentUiAlerts,
} from '../../src/database/uiAlertInfo';
import * as db from '../../src/database/connection';
import sinon from 'sinon';
Expand Down Expand Up @@ -329,4 +330,71 @@ describe('src/database/uiAlertInfo', () => {
expect(savedRecords[0].uiTestResultId).to.equal(1);
});
});

describe('checkRecentUiAlerts', () => {
it('should return a Set of keys for alerts found in the last 3 days', async () => {
// Given
const pairs = [
{ testSuiteName: 'SuiteA', individualTestName: 'Test1' },
{ testSuiteName: 'SuiteB', individualTestName: 'Test2' },
];

const mockDbRows = [
{ test_suite_name: 'SuiteA', individual_test_name: 'Test1' },
];

mockQuery.resolves(mockDbRows);

// When
const result = await checkRecentUiAlerts(pairs);

// Then
expect(mockQuery).to.have.been.calledOnce;

const sqlQuery = mockQuery.firstCall.args[0];
expect(sqlQuery).to.include("INTERVAL '3 days'");

expect(sqlQuery).to.include("('SuiteA', 'Test1')");
expect(sqlQuery).to.include("('SuiteB', 'Test2')");

expect(result).to.be.instanceOf(Set);
expect(result.size).to.equal(1);
expect(result.has('SuiteA_Test1')).to.be.true;
expect(result.has('SuiteB_Test2')).to.be.false;
});

it('should return an empty Set if no recent alerts are found', async () => {
// Given
mockQuery.resolves([]);

// When
const result = await checkRecentUiAlerts([
{ testSuiteName: 'SuiteA', individualTestName: 'Test1' },
]);

// Then
expect(result).to.be.instanceOf(Set);
expect(result.size).to.equal(0);
});

it('should handle database errors gracefully by returning an empty Set', async () => {
// Given
const consoleStub = sinon.stub(console, 'error');
mockQuery.rejects(new Error('Connection failed'));

// When
const result = await checkRecentUiAlerts([
{ testSuiteName: 'SuiteA', individualTestName: 'Test1' },
]);

// Then
expect(result).to.be.instanceOf(Set);
expect(result.size).to.equal(0);
expect(consoleStub).to.have.been.calledWith(
sinon.match('Error checking recent UI alerts')
);

consoleStub.restore();
});
});
});
48 changes: 48 additions & 0 deletions test/services/uiAlert.test.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,7 @@ describe('generateValidAlerts', () => {
let getAveragesStub: sinon.SinonStub;
let normalThresholdStub: sinon.SinonStub;
let criticalThresholdStub: sinon.SinonStub;
let checkRecentStub: sinon.SinonStub;

before(() => {
sandbox = sinon.createSandbox();
Expand All @@ -38,6 +39,7 @@ describe('generateValidAlerts', () => {
);

getAveragesStub = sandbox.stub(uiAlertInfo, 'getAverageLimitValuesFromDB');
checkRecentStub = sandbox.stub(uiAlertInfo, 'checkRecentUiAlerts');
});

beforeEach(() => {
Expand All @@ -47,6 +49,7 @@ describe('generateValidAlerts', () => {
normalThresholdStub.returns('20');
criticalThresholdStub.returns('50');
getAveragesStub.resolves({});
checkRecentStub.resolves(new Set<string>());
});

after(() => {
Expand Down Expand Up @@ -100,6 +103,51 @@ describe('generateValidAlerts', () => {
});
});

describe('recent alert filtering', () => {
const avgFirst5 = 200;
it('should skip processing if an alert was already triggered for this test in the last 3 days', async () => {
// Given
const avgNext10 = 100;
const mockAverages = {
['ComponentLoadSuite_ComponentXLoadTime']: {
avg_load_time_past_5_days: avgFirst5,
avg_load_time_6_to_15_days_ago: avgNext10,
},
};
checkRecentStub.resolves(
new Set(['ComponentLoadSuite_ComponentXLoadTime'])
);
getAveragesStub.resolves(mockAverages);

// When
const results = await generateValidAlerts([MOCK_TEST_DTO_BASE]);

// Then
expect(results).to.be.an('array').that.is.empty;
expect(getAveragesStub).to.not.have.been.called;
});

it('should process the test normally if no recent alerts exist', async () => {
// Given
const avgNext10 = 100;
const mockAverages = {
['ComponentLoadSuite_ComponentXLoadTime']: {
avg_load_time_past_5_days: avgFirst5,
avg_load_time_6_to_15_days_ago: avgNext10,
},
};
checkRecentStub.resolves(new Set());
getAveragesStub.resolves(mockAverages);

// When
const results = await generateValidAlerts([MOCK_TEST_DTO_BASE]);

// Then
expect(results).to.have.lengthOf(1);
expect(getAveragesStub).to.have.been.called;
});
});

describe('alert generation logic', () => {
const avgFirst5 = 200;

Expand Down