DB: Mongo Atlas (connector is Djongo)
Framework: Django Rest Framework
./core : main django project
./scrapy: Scrapy crawler for fetching product data
- Users
- Products
- Categories
- Explore
Clear old docker images and volumes first:
`docker-compose down` (stops containers from running, removes network and volume attachments)
`docker volume prune` (removes fs volumes that became unused from command above)
`docker-compose up --build` (rebuild web image from Dockerfile)
connect mongodb from localhost:27017 using the password in .env
you can access django app from http://localhost:8050
to run commands in a running container : docker-compose run {container_name} {command}
-
docker-compose run web python manage.py shell -
docker-compose run web sh(opens shell for the given container, it lives in /bin/sh))
TODO:
- Add pipenv
- Check Github Actions
- Profile Image Update
- Sendi mails
- Fetch Tag Products
- Integrate Scrapy
- Integrate Django Channels
python -m venv venv
./venv/bin/activate
python manage.py makemigrations
python manage.py migrate
python manage.py runserver
- https://towardsdatascience.com/web-scraping-of-10-online-shops-in-30-minutes-with-python-and-scrapy-a7f66e42446d
- https://github.com/CamilaAlvarez/scrapy-fashion-scrapers
- https://github.com/SwatiModi/e-commerce-web-scraper
- https://github.com/scrapinghub/scrapy-training
- https://github.com/vistorve/fashion-scraping
- https://github.com/CamilaAlvarez/scrapy-fashion-scrapers/blob/master/fashion_scrapper/product_loader.py
- https://github.com/PacktPublishing/Python-Web-Scraping-Cookbook
- https://github.com/PacktPublishing/Python-Web-Scraping-Second-Edition
- https://www.youtube.com/watch?v=Wp6LRijW9wg&ab_channel=HumanCode