Import csv #63

Nisha-Nathan · 2021-11-19T20:16:13Z

This PR request includes a POST request that receives a csv file that is parsed with a code written to parse csv files with certain attributes and then add the csv file to a corpus

Edited code and created new text files for TED Talks

With this code, all rows of the csv are now successfully converted into txt files, regardless of the characters in the title column. In addition, the code in auto_instances now uses the Document manager to easily and efficiently create new instances of the Document model.

Wrote new code that adds the Document instances for each TED Talk file into a Corpus object (an instance of the Corpus model).

The program no longer needs to turn reach row in the csv into its own text file. Each row of data in the csv is now directly converted into a Document, and the Documents are all combined into a Corpus.

Created a POST request that took in a csv file and returned a corpus(a serialized version of a corpus)

Wrote an API endpoint function that receives through a POST request a filename for a file in the backend (i.e. small_talks.csv). It then runs the corresponding file through parse_csv and returns as a Response a text representation of the corpus created from that file.

…_web into import_csv

backend/app/POST.py

backend/config/urls.py

backend/app/views.py

backend/app/analysis/auto_instances.py

backend/app/analysis/test.py

.idea/temp.iml

.idea/misc.xml

backend/app/views.py

backend/app/services/parse_csv.py

MBJean

Awesome work! You've tackled numerous parts of the stack and have greatly expanded the input capabilities of the project. I've left a number of comments above, a few of which will need to be addressed before we can merge this. If possible, I'd also love to see a test around parse_csv. Great work!

Cleaned up code, removed debugging statements, and deleted unnecessary files, in preparation of merge into main branch.

ADJohnson314 and others added 16 commits October 15, 2021 16:05

Spiked script parsed csv

bc43c35

Edited code and created new text files for TED Talks

Created auto_instances.py

3707310

Added Corpus creation code

3a27389

Wrote new code that adds the Document instances for each TED Talk file into a Corpus object (an instance of the Corpus model).

Streamlined process for turning csv data into a Corpus

17c64bf

The program no longer needs to turn reach row in the csv into its own text file. Each row of data in the csv is now directly converted into a Document, and the Documents are all combined into a Corpus.

POST request

2a0f288

Created a POST request that took in a csv file and returned a corpus(a serialized version of a corpus)

Delete views.py

1b8e6b0

Delete settings.py

e96f428

Delete manage.py

ee1f655

Delete __init__.py

51d4eb8

Delete asgi.py

d669d2c

Delete urls.py

9b002c4

Delete admin.py

86b6529

Merge branch 'import_csv' of https://github.com/dhmit/gender_analysis…

b627c5a

…_web into import_csv

Merge branch 'import_csv' of https://github.com/dhmit/gender_analysis…

3249fa4

…_web into import_csv