-
Notifications
You must be signed in to change notification settings - Fork 200
8477: Add opt-in memory-mapped file support for event storage #691
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
jbachorik
wants to merge
11
commits into
openjdk:master
Choose a base branch
from
jbachorik:JMC-8477
base: master
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+4,641
−56
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Created benchmark module with JMH 1.37 support - Implemented 4 benchmark suites: * EventWriteThroughputBenchmark: measures ops/sec for different event types * AllocationRateBenchmark: measures allocation rates during event writing * StringEncodingBenchmark: measures UTF-8 encoding and caching performance * ConstantPoolBenchmark: measures constant pool buildup and lookup - Baseline results (Apple M1 Max, JDK 21.0.5): * Simple events: 986K ops/s * Multi-field events: 862K ops/s * Unique strings: 11.9x slower than repeated strings * Identified OutOfMemoryError with unbounded constant pool growth - Comprehensive documentation with build/run instructions - JSON results for programmatic comparison 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Implements opt-in memory-mapped file (mmap) support for off-heap event storage to reduce heap pressure during JFR recording. The implementation uses double-buffered per-thread mapped byte buffers with automatic rotation and background flushing. Key Components: - LEB128MappedWriter: Fixed-size mmap-backed LEB128 writer with overflow detection - ThreadMmapManager: Manages per-thread double buffers and coordinates rotation with background flush executor (daemon threads) - Chunk: Enhanced with automatic rotation support when buffer fills - RecordingImpl: Conditionally uses mmap or heap mode based on system properties Configuration: - org.openjdk.jmc.flightrecorder.writer.mmap.enabled=true (default: false) - org.openjdk.jmc.flightrecorder.writer.mmap.chunkSize=<bytes> (default: 4MB) Performance: Benchmarks show 8-12% improvement over heap mode across all event types (writeSimpleEvent +8.3%, writeMultiFieldEvent +11.9%, writeRepeatedStringsEvent +11.9%, writeStringHeavyEvent +10.4%). Backward Compatibility: Heap mode remains the default. All existing tests pass. New unit tests and integration tests verify mmap functionality. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Replace system property-based configuration with fluent builder API for configuring memory-mapped file settings when creating JFR recordings. Changes: - Add RecordingSettingsBuilder.withMmap() methods for enabling mmap with default (4MB) or custom chunk sizes - Add useMmap() and getMmapChunkSize() methods to RecordingSettings - Remove system property constants from RecordingImpl: - org.openjdk.jmc.flightrecorder.writer.mmap.enabled - org.openjdk.jmc.flightrecorder.writer.mmap.chunkSize - Update MmapRecordingIntegrationTest to use builder pattern instead of system properties - Add @SInCE 10.0.0 javadoc tags to all new public API methods Migration example: Before: System.setProperty("org.openjdk.jmc.flightrecorder.writer.mmap.enabled", "true") After: Recordings.newRecording(stream, settings -> settings.withMmap()) 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
Fix maven-jar-plugin configuration to correctly locate MANIFEST.MF from
the resources directory, enabling successful benchmark JAR creation.
Changes:
- Add manifestFile configuration pointing to ${project.build.outputDirectory}/META-INF/MANIFEST.MF
- Create comprehensive README.md with:
- Build and usage instructions
- All 13 benchmark descriptions
- Profiling guide with examples
- Output format options (JSON, CSV, text)
- Common JMH options reference
- Troubleshooting guide
- Add src/main/resources/META-INF/MANIFEST.MF for JAR packaging
The benchmark JAR (benchmarks.jar) now builds successfully via 'mvn clean package'
and supports all JMH command-line arguments out-of-the-box.
Verified functionality:
- JAR executes and responds to -h (help)
- All 13 benchmarks listed with -l
- Benchmarks execute successfully (e.g., writeSimpleEvent: ~950K ops/s)
- JSON/CSV output formats work
- Profilers available (gc, jfr, stack, etc.)
- Regex filtering and parameterization work
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Co-Authored-By: Claude <noreply@anthropic.com>
Add compare.py Python script for easy comparison of JMH benchmark results between baseline and optimized runs. The script automatically handles different benchmark modes (throughput vs average time) and displays improvements with directional indicators. Changes: - Add compare.py utility script with: - Automatic detection of benchmark modes (thrpt, avgt, etc.) - Correct calculation of improvements (higher/lower is better) - Formatted output with directional arrows (↑/↓) - Support for parameterized benchmarks - Optional custom title for comparisons - Update README.md with: - Detailed compare.py usage instructions - Example output showing improvement percentages - Explanation of benchmark mode handling Usage: python3 compare.py baseline.json optimized.json "Optional Title" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
|
👋 Welcome back jbachorik! A progress list of the required criteria for merging this PR into |
|
❗ This change is not yet ready to be integrated. |
The benchmark module contains JMH benchmark classes, not JUnit tests. Configure maven-surefire-plugin to skip test execution for this module. Fixes CI failure: 'No tests to run' error
Replace Float.floatToIntBits/Double.doubleToLongBits conversions with direct buffer.putFloat/putDouble calls. This is more idiomatic and eliminates unnecessary bit conversion overhead. Changes: - writeFloat(): Use buffer.putFloat() instead of writeIntRaw(floatToIntBits()) - writeDouble(): Use buffer.putDouble() instead of writeLongRaw(doubleToLongBits()) Benefits: - More readable code (intent is clearer) - Potentially faster (avoids bit conversion) - Consistent with standard ByteBuffer API usage All 322 tests pass.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
!WARNING!
This PR depends on #690 - it will stay in draft until the other PR is merged.
Summary
Implements JMC-8477: Add opt-in memory-mapped file support to JFR Writer API for off-heap event storage.
Problem
The current JFR Writer API stores all event data in on-heap byte arrays during recording creation, causing:
Solution
This PR adds opt-in memory-mapped file support, allowing event data to be stored off-heap.
API
Users opt in via the builder pattern:
Benefits
Performance Results
JMH benchmarks show consistent improvements across event types:
Backward Compatibility
Testing
Commits
Documentation
🤖 Generated with Claude Code
Progress
Integration blocker
Issue
Reviewing
Using
gitCheckout this PR locally:
$ git fetch https://git.openjdk.org/jmc.git pull/691/head:pull/691$ git checkout pull/691Update a local copy of the PR:
$ git checkout pull/691$ git pull https://git.openjdk.org/jmc.git pull/691/headUsing Skara CLI tools
Checkout this PR locally:
$ git pr checkout 691View PR using the GUI difftool:
$ git pr show -t 691Using diff file
Download this PR as a diff file:
https://git.openjdk.org/jmc/pull/691.diff
Using Webrev
Link to Webrev Comment