Skip to content

Conversation

@jzunigax2
Copy link

Optimize file uploads with targeted existence check and rename-based updates

Previously:

  • Update() called NewObject() to check file existence
  • NewObject() used folders.ListAllFiles() - fetches ALL files in directory to find one matching file
  • Inefficient for directories with many files
  • Delete-then-upload pattern: old file deleted before new upload starts
  • If upload fails, data is lost

Optimizations

  1. Targeted Existence Check
  • New preUploadCheck() uses files.CheckFilesExistence() API
  • Checks for specific file by name instead of listing entire directory, plus previous approach was unreliable in the case the response had to be paginated
  1. Rename-Based Atomic Updates
  • Rename existing file to backup with random suffix
  • Upload new file with original name
  • On success: delete backup; on failure: restore backup
  • Guarantees data safety during upload failures

@jzunigax2 jzunigax2 requested a review from larryrider December 22, 2025 21:52
@jzunigax2 jzunigax2 self-assigned this Dec 22, 2025
@jzunigax2 jzunigax2 changed the title Refactor/update function and existance check [PB-5606]: refactor/update function and existance check Dec 22, 2025
@jzunigax2 jzunigax2 merged commit 551fbc9 into master Dec 24, 2025
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants