Image Dataloader with Field type #18

kartikdutt18 · 2020-06-05T14:07:19Z

This is nearly the same PR as the other image dataloader. Difference is that I have used field vector inserting bounding boxes.
So in preprocessing of dataset for model we would have instant lookups.

dataloader/dataloader.hpp

zoq · 2020-06-05T21:14:42Z

This is nearly the same PR as the other image dataloader. Difference lies that I have used field vector inserting bounding boxes.
So in preprocessing of dataset for model we would have instant lookups.

So I guess we can close the other one?

KimSangYeon-DGU · 2020-06-05T22:01:20Z

@zoq Yes, we can close the other one if this is better :) We're currently seeing if which approach is better between this and #13.

KimSangYeon-DGU

Left some style and possible improvement comments, it looks good to go :)

augmentation/augmentation.hpp

dataloader/dataloader.hpp

dataloader/dataloader_impl.hpp

dataloader/dataloader.hpp

dataloader/dataloader_impl.hpp

utils/utils.hpp

KimSangYeon-DGU

Another comment

dataloader/dataloader.hpp

saksham189

Just some minor comments. Looks great!

augmentation/augmentation.hpp

dataloader/dataloader.hpp

KimSangYeon-DGU

I think it's almost ready to be merged, please check the comments including the source path of serverName in the CurlDownloadTest. I mean this.

augmentation/augmentation.hpp

dataloader/dataloader.hpp

augmentation/augmentation.hpp

dataloader/datasets.hpp

augmentation/augmentation.hpp

augmentation/augmentation_impl.hpp

zoq

Really nice to see this come to live, do you think we could provide a simple notebook to show how this code can be used to augment some image dataset?

.travis.yml

augmentation/augmentation.hpp

augmentation/augmentation_impl.hpp

dataloader/dataloader.hpp

dataloader/dataloader_impl.hpp

kartikdutt18 · 2020-06-10T06:03:43Z

do you think we could provide a simple notebook to show how this code can be used to augment some image dataset?

Sure I can give it a try, This would be for examples repo or model repo? Thanks.

KimSangYeon-DGU

Looks great! I left some minor comments :)

tests/dataloader_tests.cpp

KimSangYeon-DGU · 2020-06-11T06:25:28Z

@kartikdutt18 I think this is ready to be merged. Can you squash the commits to be concise?

kartikdutt18 · 2020-06-11T06:26:49Z

Sure. Let me do that right now.

Trained a lenet1 model Add All Weights Add unzip utility function, fix linux build Completed everything Fix build, Style fixes next Use force local for windows while unzipping tar files Use force local for windows while unzipping tar files Added Utility Function Added Utility Function Style Fix Fix Typo causing build error in windows Fix Typo causing build error in windows Fix const issue for windows Extract in data folder Reposition force local This should work Print Path in windows for debugging Print Path in windows for debugging Print Path in windows for debugging Stip components of tar Stip components of tar strip componenets Initial Definition Add List Dir utility function Add List Dir utility function Squash this, annotation read Yay, we can read xml now Complete labels portion of dataloader Style Fix Style Fix Style Fix Add basic definition of augmentation class Load images according to annotation in column format, Reduce complexity by preventing whole iteration Augmentation work Augmentation work

Style fixes Style fixes

KimSangYeon-DGU

Amazing! Looks good to me 👍

kartikdutt18 · 2020-06-11T12:21:23Z

Pushing a commit to delete the dataset.

Upload dataset temp Resolve merge conflicts Resolve merge conflicts Resolve merge conflicts Resolve merge conflicts Style fixes Set up constructor Complete Dataloader, write function definition for image dataloader Build Fixed, Style Fixed style fixed Fix linux and Windows build by ignoring invalid files like .DS_Store Add image dataloader, needs clean up Style fixes Check for file type Check for correct extention and filetype Style Fixed Build Fixed, Nice way to avoid hidden files Style Fixed Clean Up Why is config there even though I did a rebase Add data split (internal), need implementation only for field type Style Fixed need to seperate the declaration Fixed style fixes, commits also need clean up Remove template from Augmentation Fixed the duplicacy error Add mat type support (there is invalid read) Mat type gives invalid read Hmm, style fixes, commits also need clean up Typo -> invalid read -> Fixed Remove extra lines Rename cell to image Style Fixes and use mlpack URL Make augmentation case insensitive Allow multiple Resize Transform Add unknown augmentation warning Style Fixes and boundary checks Increase dataset size, adjust comments, change URL Style Fixes and boundary checks Remove locally held datasets

saksham189

Thanks for the work on this! LGTM

zoq · 2020-06-11T21:09:40Z

do you think we could provide a simple notebook to show how this code can be used to augment some image dataset?

Sure I can give it a try, This would be for examples repo or model repo? Thanks.

I think it makes sense to put it into the examples repository, super excited to see this merged.

KimSangYeon-DGU · 2020-06-11T22:18:05Z

@kartikdutt18 Thanks for the amazing work!

kartikdutt18 · 2020-06-12T02:48:35Z

Thanks a lot @KimSangYeon-DGU, @zoq, @saksham189 for the reviews and all the help.

mlpack-bot bot added s: needs review s: unanswered s: unlabeled labels Jun 5, 2020

kartikdutt18 commented Jun 5, 2020

View reviewed changes

dataloader/dataloader.hpp Outdated Show resolved Hide resolved

zoq added t: added feature and removed s: unanswered s: unlabeled labels Jun 5, 2020

KimSangYeon-DGU mentioned this pull request Jun 5, 2020

Add Image Dataloader for PASCAL VOC and CIFAR 10 dataset. #13

Closed

6 tasks

KimSangYeon-DGU reviewed Jun 5, 2020

View reviewed changes

dataloader/dataloader.hpp Outdated Show resolved Hide resolved

saksham189 reviewed Jun 7, 2020

View reviewed changes

augmentation/augmentation.hpp Outdated Show resolved Hide resolved

dataloader/dataloader.hpp Outdated Show resolved Hide resolved

kartikdutt18 force-pushed the ImageDataloadersBetter branch from 5b8e0c9 to 1218f53 Compare June 8, 2020 08:24

KimSangYeon-DGU reviewed Jun 8, 2020

View reviewed changes

dataloader/datasets.hpp Outdated Show resolved Hide resolved

saksham189 reviewed Jun 8, 2020

View reviewed changes

augmentation/augmentation.hpp Show resolved Hide resolved

saksham189 reviewed Jun 8, 2020

View reviewed changes

augmentation/augmentation_impl.hpp Outdated Show resolved Hide resolved

augmentation/augmentation_impl.hpp Outdated Show resolved Hide resolved

zoq reviewed Jun 9, 2020

View reviewed changes

KimSangYeon-DGU reviewed Jun 11, 2020

View reviewed changes

kartikdutt18 force-pushed the ImageDataloadersBetter branch from b0a8c2c to 022ba45 Compare June 11, 2020 06:00

kartikdutt18 added 3 commits June 11, 2020 11:59

Complete LoadObjectDetection Function

6b306aa

Style fixes Style fixes

Add tests for augmentation

a975740

kartikdutt18 force-pushed the ImageDataloadersBetter branch from 022ba45 to c859e8e Compare June 11, 2020 06:31

KimSangYeon-DGU approved these changes Jun 11, 2020

View reviewed changes

kartikdutt18 force-pushed the ImageDataloadersBetter branch from c859e8e to 3fc6e7b Compare June 11, 2020 12:29

saksham189 approved these changes Jun 11, 2020

View reviewed changes

mlpack-bot bot removed the s: needs review label Jun 11, 2020

KimSangYeon-DGU merged commit 5ea20fc into mlpack:master Jun 11, 2020

Image Dataloader with Field type #18

Image Dataloader with Field type #18

Uh oh!

Conversation

kartikdutt18 commented Jun 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

zoq commented Jun 5, 2020

Uh oh!

KimSangYeon-DGU commented Jun 5, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KimSangYeon-DGU left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KimSangYeon-DGU left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

saksham189 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

KimSangYeon-DGU left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zoq left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kartikdutt18 commented Jun 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

KimSangYeon-DGU left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KimSangYeon-DGU commented Jun 11, 2020

Uh oh!

kartikdutt18 commented Jun 11, 2020

Uh oh!

KimSangYeon-DGU left a comment

Choose a reason for hiding this comment

Uh oh!

kartikdutt18 commented Jun 11, 2020

kartikdutt18 commented Jun 5, 2020 •

edited

Loading

KimSangYeon-DGU commented Jun 5, 2020 •

edited

Loading

KimSangYeon-DGU left a comment •

edited

Loading

KimSangYeon-DGU left a comment •

edited

Loading

kartikdutt18 commented Jun 10, 2020 •

edited

Loading