Users/tanmayapanda/address review comments#458
Merged
ag-ramachandran merged 3 commits intofeature/IngestV2from Jan 20, 2026
Merged
Users/tanmayapanda/address review comments#458ag-ramachandran merged 3 commits intofeature/IngestV2from
ag-ramachandran merged 3 commits intofeature/IngestV2from
Conversation
ag-ramachandran
requested changes
Jan 20, 2026
| * Atomic counter for round-robin container selection. Increments on each | ||
| * upload to distribute load evenly across containers. | ||
| */ | ||
| private val containerIndexCounter = AtomicInteger(0) |
Contributor
There was a problem hiding this comment.
AtomicLong perhaps ?
Collaborator
Author
There was a problem hiding this comment.
I thought of so but AtomicLong seems to be an overkill perhaps, since AtomicInteger has already around 2.1 billion positive values. Added Math.floorMod to handle the negative values incase overflow occurs.
ingest-v2/src/main/kotlin/com/microsoft/azure/kusto/ingest/v2/uploader/ContainerUploaderBase.kt
Show resolved
Hide resolved
ingest-v2/src/main/kotlin/com/microsoft/azure/kusto/ingest/v2/uploader/ICustomUploader.kt
Show resolved
Hide resolved
quickstart/src/main/java/com/microsoft/azure/kusto/quickstart/SampleApp.java
Show resolved
Hide resolved
ag-ramachandran
approved these changes
Jan 20, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull request introduces significant changes to the ingestion client and uploader logic to simplify and clarify multi-source ingestion, enforce stricter type safety, and improve concurrency handling. The main focus is to restrict multi-source ingestion to only support blob sources, remove internal logic for uploading local sources within multi-ingest, and enhance the uploader's parallelism and load distribution.
Multi-source ingestion API and logic changes:
ingestAsync) to only acceptBlobSourceobjects, both in the interface and implementation, and updates all related documentation to clarify that local sources must be uploaded separately before ingestion. This prevents partial failure scenarios and enforces a clearer, safer workflow. [1] [2] [3] [4]uploadLocalSourcesAsync) fromQueuedIngestClient, making it the caller's responsibility to upload local files/streams before calling multi-ingest. [1] [2]Error handling and messaging improvements:
IngestClientExceptionto use themessageparameter for clarity and consistency. [1] [2]Uploader concurrency and load balancing enhancements:
ContainerUploaderBasefrom random to round-robin using an atomic counter, ensuring more even load distribution across containers during uploads and retries. [1] [2] [3]Java interop and API usability:
DefaultConfigurationCachefor easier creation from Java, supporting both minimal and full-parameter configurations.@JvmOverloadsconstructors toFileSourceandStreamSourcefor better Java interoperability and sensible defaults. [1] [2]These changes collectively make the ingestion API safer, simpler, and more robust, especially in multi-source scenarios.### Added
Changed
Fixed