Renewable Energy Integration and Load Forecasting in Green Cloud Infrastructure

Industry package for building a production-grade pipeline that:

• Ingests grid & renewable generation signals (solar/wind), cloud workload metrics, and carbon intensity.
• Trains time-series forecasting models (LSTM, baseline statistical models) for load and renewables.
• Exposes real-time inference via a FastAPI microservice.
• Orchestrates with Docker, deploys to Kubernetes with manifests.
• Provides monitoring hooks (Prometheus metrics, Grafana dashboard stub).
• Includes cost/carbon-aware scheduling signal for green workload shifting.

Tech: Python · PyTorch · scikit-learn · FastAPI · Docker · K8s · Prometheus

---

Architecture

Data Sources (Grid API, Carbon API, Workload Metrics) --> ETL (src/pipelines/)
  --> Feature Store (parquet) --> Training (src/models/) --> Registry (models/)
  --> Inference API (src/service/) --> Signals (src/signals/) --> Schedulers / Autoscalers
                                             |--> Monitoring (Prometheus)

---

Quickstart (Local)

python3 -m venv .venv && source .venv/bin/activate
pip install -r requirements.txt

# 1) Generate synthetic data & train models
python -m src.pipelines.generate_synthetic --out data/raw
python -m src.pipelines.feature_engineer --inp data/raw --out data/processed
python -m src.models.train --config configs/train_lstm.yaml

# 2) Run inference service
uvicorn src.service.api:app --reload --port 8000

# 3) Query prediction
curl -s 'http://localhost:8000/predict?horizon=24'

---

Docker & K8s

# Build image
docker build -t green-cloud-forecasting:latest .

# Run with Docker Compose (includes Prometheus stub)
docker compose up

# K8s (manifests)
kubectl apply -f k8s/

---

Repository Layout

green-cloud-forecasting/
├── configs/                 # Training/configuration YAMLs
├── data/                    # (created) raw/processed datasets
├── docs/                    # System docs & dashboards
├── k8s/                     # Manifests for API + Prometheus ServiceMonitor stub
├── models/                  # Saved model checkpoints & metadata
├── src/
│   ├── integrations/        # API clients (carbon, grid, workload) - with safe mocks
│   ├── pipelines/           # ETL and feature engineering
│   ├── models/              # LSTM & baselines; trainer CLI
│   ├── service/             # FastAPI app for inference with Prometheus metrics
│   ├── signals/             # Carbon-aware scheduling score
│   └── utils/               # IO, config, logging helpers
├── tests/                   # Minimal unit tests
├── .github/workflows/ci.yml # Lint & tests
├── Dockerfile
├── docker-compose.yaml
├── Makefile
├── requirements.txt
├── LICENSE
└── CITATION.cff