Skip to content

Conversation

@kabragaurav
Copy link
Collaborator

@kabragaurav kabragaurav commented Dec 11, 2025

Issues

When we see logs in Kusto, we see only request paths but query params (if any) were being removed.

helixacm_service_logs
| where timestamp >= ago(1d)
| where message has "AuditLog"
| project timestamp, message, appData, defaultLog__logger, level
| top 20 by timestamp asc
image
  • My PR addresses the following Helix issues and references them in the PR description: CICP-3095

Description

  • Here are some details about my PR:
    AuditLogFilter.java sets value for AUDIT_LOG key.
Screenshot 2025-12-11 at 1 47 33 PM

Which is passed to logger.write() method.

Screenshot 2025-12-11 at 1 47 56 PM

This method is overwritten by HelixRestAuditLogger and DefaultHelixRestAuditLogger.java.

Screenshot 2025-12-11 at 1 48 52 PM

Tests

  • The following tests are written for this issue:

TestClusterAccessor.testAuditLogIncludesQueryParams

  • The following is the result of the "mvn test" command on the appropriate module:

mvn test -pl helix-rest -Dtest=TestClusterAccessor#testAuditLogIncludesQueryParams 2>&1

Screenshot 2025-12-11 at 1 53 20 PM

Estimation for Log Volume Increase

helixacm_service_logs
| where timestamp >= ago(1d)
| where message has "AuditLog"
| count 

556376798

helixacm_service_logs
| where timestamp >= ago(1d)
| count 

1526658348

So AuditLog consists of roughly 36.4%.

By Paretto principle of 80-20 say 80% of requests have query parameters.
And also assuming having query params increases request path length by 40 chars (In UTF-8 if a char is 1B then 40B per request path).

Volume Increase Calculation

Daily AuditLog entries:           556,376,798
Requests with query params (80%): 445,101,438
Average query string size increase:        40 bytes

Additional data per day:
445,101,438 × 40 bytes ~ 18 GB/day

Considering disk compression of 5x: 3.6 GB/day

Copy link
Collaborator

@proud-parselmouth proud-parselmouth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just 1 comment.


// Verify that query parameters are included in the request path
String requestPath = auditLog.getRequestPath();
Assert.assertTrue(requestPath.contains("?"),
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should we just write one assert testing if the auditLog.getRequestPath() matches with the expected requested path.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed!

Copy link
Collaborator

@proud-parselmouth proud-parselmouth left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM.

Copy link
Collaborator

@bellatrix007 bellatrix007 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good, but can we remove the internal LinkedIn related info like CICP ticket number in the PR description.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants