[improve][client]: add --streaming option to topics partitioned-stats-internal#23380
[improve][client]: add --streaming option to topics partitioned-stats-internal#23380dlg99 wants to merge 4 commits intoapache:masterfrom
Conversation
…-internal (apache#261) (cherry picked from commit 471669a)
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #23380 +/- ##
============================================
+ Coverage 73.57% 74.52% +0.94%
- Complexity 32624 34468 +1844
============================================
Files 1877 1934 +57
Lines 139502 145158 +5656
Branches 15299 15874 +575
============================================
+ Hits 102638 108176 +5538
+ Misses 28908 28679 -229
- Partials 7956 8303 +347
Flags with carried forward coverage won't be shown. Click here to find out more.
|
lhotari
left a comment
There was a problem hiding this comment.
Instead of an InputStream, a Reader would be more appropriate for the interface.
The reason for this is in this case, the payload is most likely UTF-8 encoded which uses multi-byte characters for most non-ascii characters.
When reading in 2048 chunks, a multi-byte character might get split at the boundary which will corrupt the output.
Using a Reader would address this without any additional logic.
A long time ago I worked on such a fix for Gradle, https://issues.gradle.org/browse/GRADLE-3329. In that case there was custom buffering and fixing the issue wasn't trivial. I ended up using java.nio.charset.CharsetDecoder and java.nio.charset.CoderResult#isUnderflow to detect when a byte is missing.
A Reader will buffer the underlying bytes when there's a multi-byte character case and the remaining byte is not yet readable. That's why it's better to rely on a Reader in the interface.
|
slightly related #18263 |
|
@dlg99 please address the review feedback, #23380 (review). |
Motivation
Streaming the stats output directly to the standard output without parsing the response in memory.
Modifications
added
--streamingflag fortopics stats-internal/partitioned-stats-internalcommands to stream the stats.Verifying this change
This change added unit tests.
Does this pull request potentially affect one of the following parts:
added
--streamingflag fortopics stats-internal/partitioned-stats-internalcommandsDocumentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: dlg99#21