Skip to content

Feat/downloadwroker optimizations #1798

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
guruhubb wants to merge 67 commits into
base: staging
Choose a base branch
from
Open

Feat/downloadwroker optimizations #1798

guruhubb wants to merge 67 commits into from

Conversation

@guruhubb
Copy link
Member

@guruhubb guruhubb commented Dec 27, 2025

Changes

Fixes

Tests

Tasks to complete before merging PR:

  • Ensure system tests are passing. If not Run them manually to check for any regressions 📋
  • Do any new system tests need added to test this change? do any existing system tests need updated? If so create a PR at 0chain/system_test
  • Merge your system tests PR to master AFTER merging this PR

Associated PRs (Link as appropriate):

  • blobber:
  • 0chain:
  • system_test:
  • zboxcli:
  • zwalletcli:
  • Other: ...

Hitenjain14 added 30 commits July 22, 2024 03:17
@Hitenjain14
add version marker and block hasher
f9eb675
@Hitenjain14
add repair info in commit worker
02e203d
@Hitenjain14
add check for repair marker
c981f06
@Hitenjain14
add allocation version and check for directory
b5cbb79
@Hitenjain14
prefix delete
526a117
@Hitenjain14
fix consensus in delete
6486442
@Hitenjain14
fix consensus thresh check
f9f8237
@Hitenjain14
fix path level in delete
e015027
@Hitenjain14
remove vt from downloadworker
a8bf8bb
@Hitenjain14
rm vt in reader
b91b592
@Hitenjain14
Merge branch 'feat/enterprise-blobber' of https://github.com/0chain/g…
9beaccc 
…osdk into feat/enterprise-blobber
@Hitenjain14
remove single blobber for single client mode
f5b51c0
@Hitenjain14
add check for nil ref
4d33f27
@Hitenjain14
remove mask reset
796de38
@Hitenjain14
check change count
da1aa55
@Hitenjain14
chane ref check
a9814df
@Hitenjain14
remove validation root from Ref
08de511
@Hitenjain14
add list logs
e7df719
@Hitenjain14
Merge branch 'feat/enterprise-blobber' of https://github.com/0chain/g…
3f00405 
…osdk into feat/enterprise-blobber
@Hitenjain14
add response
c2e8b89
@Hitenjain14
add debug logs
4496ab6
@Hitenjain14
add log for selected file ref
daa353d
@Hitenjain14
add check for rate limit
974959f
@Hitenjain14
remove get ref log
ec1a470
@Hitenjain14
add repair for enterprise blobbers
facdb4f
@Hitenjain14
add delete for dir remotepath in move
d15f3ce
@Hitenjain14
fix no change check
f2bb064
@Hitenjain14
fix consensus and add check for no change err
b0e385f
@Hitenjain14
fix list completed repair
a2081f3
@Hitenjain14
use dir as destpath
af515ba
Hitenjain14 and others added 30 commits August 13, 2024 11:24
@Hitenjain14
check vm
645ded9
@Hitenjain14
move upload success to debug
007b982
@Hitenjain14
Merge branch 'sprint-1.17' into feat/enterprise-blobber
b293286
@Hitenjain14
Merge branch 'sprint-1.17' of https://github.com/0chain/gosdk into fe…
a9a39a9 
…at/enterprise-blobber
@Hitenjain14
fix nested move and rename
2119c4a
@Hitenjain14
fix nested rename
6648ada
@Hitenjain14
rmv log for invalid path
acf47eb
@Hitenjain14
Merge branch 'sprint-1.17' of https://github.com/0chain/gosdk into fe…
970cd10 
…at/enterprise-blobber
@Hitenjain14
Merge remote-tracking branch 'origin' into feat/enterprise-blobber
50adab4
@Hitenjain14
Merge remote-tracking branch 'origin' into feat/enterprise-blobber
75bab85
@Hitenjain14
Merge branch 'staging' into feat/enterprise-blobber
f71928d
@Hitenjain14
Merge branch 'staging' into feat/enterprise-blobber
c8f84a9
@Jayashsatolia403
Merge branch 'staging' into feat/enterprise-blobber
aaf11db
@Hitenjain14
Merge remote-tracking branch 'origin' into feat/enterprise-blobber
271ccc6
@Hitenjain14
comment test
237970b
@Hitenjain14
Merge branch 'feat/enterprise-blobber' of https://github.com/0chain/g…
43dcb10 
…osdk into feat/enterprise-blobber
@Hitenjain14
fix release of buffer
d4d99f5
@Hitenjain14
change cdn and wasm path
655bacd
@Hitenjain14
update go version in make wasm
f967f8a
@Hitenjain14
add method for wasm type
2f06303
@Hitenjain14
set wasm type in window object
d8f100c
@Hitenjain14
merge file write and callback changes
2f3d040
@Hitenjain14
fix build
0fe81ae
@Hitenjain14
merge changes
ed9fbc8
@Hitenjain14
change init signature
3952a35
@Hitenjain14
fix windows build
d9e880d
@Hitenjain14
add check for wallet id
a9e5b9a
@Hitenjain14
add check and change transport
e3d0b1d
feat: optimize download worker performance
5d46eb4 
- Increase channel buffer sizes (3x worker count) for better throughput
- Parallelize erasure decoding for multiple blocks
- Increase response channel buffer sizes
- Improve download performance without breaking data integrity

Compatible with PR-177 and PR-193 baseline (commit e3d0b1d)
Increase HTTP connection pool limits for high-concurrency workloads
3c12947 
- Increase MaxIdleConnsPerHost from 100 to 2048
- Increase MaxIdleConns from 500 to 5000
- Fixes 'no free connections available' error during warp tests
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@guruhubb @Hitenjain14 @Jayashsatolia403