Skip to content

This is a large file processor which deals with files which are bigger than the jvm memory. This sorts the text contents of the original file and write into a different file using external sorting techniques.

Notifications You must be signed in to change notification settings

sebin7tony/LargeFileProcessor

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

5 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

LargeFileProcessor

This is a large file processor which deals with files which are bigger than the jvm memory. This sorts the text contents of the original file and write into a different file using external sorting techniques.

Data

Input data has to be placed under src\main\resources\data\ folder and output data will be get generated under src\main\resources\output\ folder.

Test data are from https://www.kaggle.com/freecodecamp/all-posts-public-main-chatroom

Test 1 : freecodecamp_casual_chatroom_anon.csv (1 GB)

Test 2 : freecodecamp_casual_chatroom.csv (2.6 GB)

About

This is a large file processor which deals with files which are bigger than the jvm memory. This sorts the text contents of the original file and write into a different file using external sorting techniques.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages