MapReduce

Map Reduce project for Engineering Distributed Infrastructure class at University of Warsaw

Link to initial project design docs.

Set up application on Google Cloud Platform with Kubernetes

Using script

1. Create a new project on Google Cloud Platform
2. Install Google Cloud SDK
   • Google Cloud SDK
3. Authenticate with Google Cloud SDK
   • gcloud auth login
   • gcloud config set project PROJECT_ID
4. Run the setup script
   • src/scripts/setup_app_on_gcloud.sh

Useful links

• Workloads (pods)
• GKE Storage
• NFS instance

Using application deployed to GKE on Google Cloud Platform

1. Use GKE feature of port forwarding to a specific pod:
   • kubectl port-forward mapreduce-upload 8000:8000
   • Open browser and go to localhost:8000
2. After running setup_app_on_gcloud.sh script You should see the message:
   • MapReduce is now available at: http://<Address> - just go to that address
3. Check the address of the service:
   • kubectl get svc mapreduce-upload-loadbalancer -o jsonpath='{.status.loadBalancer.ingress[0].ip}'
4. To check for logs go to Logging --> Log-based metrics