Feat/docs #176

shawnpetros · 2025-12-12T17:42:17Z

this is missing the data package details with Athena and whatnot which I'm sure are documented in that repo

#125 address the doc update
#22 backend doc
#23 frontend doc
#72 update repo doc
#73 Batch document issue

Copilot

Pull request overview

This PR restructures and significantly improves the WindWatts project documentation by creating a centralized documentation hub with organized guides for different audiences (developers, data scientists, DevOps). The changes transform a monolithic README into a well-organized documentation structure with dedicated guides for quickstart, backend development, frontend development, data science, and deployment.

Key Changes

Created a new docs/ directory with 6 comprehensive markdown files covering different aspects of the project
Simplified the main README to provide a high-level overview with links to detailed documentation
Added a CONTRIBUTING.md guide with clear contribution guidelines and standards
Updated component-level READMEs to reference centralized documentation

Reviewed changes

Copilot reviewed 11 out of 12 changed files in this pull request and generated 11 comments.

Show a summary per file

File	Description
README.md	Significantly simplified to provide high-level overview and quick links to detailed documentation
CONTRIBUTING.md	New file establishing contribution guidelines, branch naming conventions, and coding standards
docs/README.md	New documentation hub with table of contents linking to all documentation sections
docs/01-overview.md	New file documenting system architecture, tech stack, and component relationships
docs/02-quickstart.md	New comprehensive quickstart guide with Docker and native setup instructions
docs/03-backend.md	New backend development guide covering FastAPI setup, database configuration, and testing
docs/04-frontend.md	New frontend development guide for React/Vite setup and development workflow
docs/05-data-science.md	New data science guide covering dw_tap package, ERA5 data details, and scientific background
docs/06-deployment.md	New deployment guide for containerized and production deployments
windwatts-ui/README.md	Updated to reference centralized frontend documentation
windwatts-api/README.md	Updated to reference centralized backend documentation
.gitignore	Added entries to ignore Cursor thoughts directory

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

docs/02-quickstart.md

docs/01-overview.md

docs/05-data-science.md

README.md

docs/06-deployment.md

Copilot · 2025-12-12T17:47:43Z

docs/06-deployment.md

+### Build
+
+```shell
+docker build -t tap-api:latest .


The example references 'tap-api' as the image name, but the project is called 'WindWatts'. Consider using a more consistent naming convention like 'windwatts-api' to match the project structure.

@copilot open a new pull request to apply changes based on this feedback

docs/06-deployment.md

Copilot · 2025-12-12T17:47:43Z

docs/03-backend.md

+```bash
+# Example for installing the wheel manually (check Dockerfile for exact URL/version)
+curl -O https://windwatts-era5.s3.us-west-2.amazonaws.com/windwatts_data-1.0.4-py3-none-any.whl
+pip install windwatts_data-1.0.4-py3-none-any.whl
+```


The instructions here download and install the windwatts_data wheel directly from an S3 URL using curl and pip install without any integrity verification. If an attacker can modify the S3 object or the URL ever points to a compromised file, developers following these steps could unknowingly execute arbitrary code in their environment. Consider publishing a checksum or signature for the wheel and documenting how to verify it before installation, or distributing the package via a channel that provides built-in integrity verification (e.g., a private PyPI index or signed artifacts).

@copilot open a new pull request to apply changes based on this feedback

RLiNREL

Overall, good documentations! It seems some of the outdated files were picked up, such as, dw_tap and api.py. Those contents should be removed and updated to match the current implementations.

RLiNREL · 2025-12-18T19:43:44Z

README.md

-*Development* is the default mode (run if no flag is specified). In contrast, you can run the endpoint in the *Production* mode using: `python api.py --production` or `python api.py -p` -- this is what is used in the container deployment described above (to see the details, check the end of `Dockerfile`).
-
-To see how these production and development modes are configured, refer to `config.json` and see what `host` and `port` values are specified.  
+WindWatts is a distributed wind resource assessment tool developed by NREL. It provides processed wind speed data, energy production estimates, and visualizations for locations across the United States.


"developed by NLR (formerly NREL)"

docs/05-data-science.md

RLiNREL · 2025-12-18T21:59:46Z

docs/README.md

+2.  [Quickstart](02-quickstart.md) - Get the app running in < 15 minutes.
+3.  [Backend Guide](03-backend.md) - API development.
+4.  [Frontend Guide](04-frontend.md) - UI development.
+5.  [Data Science](05-data-science.md) - Core logic and ERA5 data.


similarly, remove the Data Science part

RLiNREL · 2025-12-18T22:03:50Z

docs/05-deployment.md

the deployment instruction should be updated as we are moving to docker compose from single docker container for the old app

here is draft for the current set up using docker compose, the docker build command for Flask app is outdated.

# Deployment Guide ## Local Development Deployment Uses Docker Compose to run all services (API, UI, database, reverse proxy). ### Prerequisites 1. Docker and Docker Compose installed 2. AWS credentials with access to WindWatts data 3. Google Maps API key and Map ID ### Configuration **Root `.env`:** ```plaintext WINDWATTS_DATA_URL=<s3-url-for-windwatts-data-package> AWS_ACCESS_KEY_ID=<your-access-key> AWS_SECRET_ACCESS_KEY=<your-secret-key> AWS_SESSION_TOKEN=<your-session-token> WINDWATTS_DATA_CONFIG_SECRETS_ARN=<your-secrets-arn>

.env.development:

VITE_API_BASE_URL=http://windwatts-proxy:80 VITE_MAP_API_KEY=<your-map-api-key> VITE_MAP_ID=<your-map-id>

windwatts_data_config.json:

Configure AWS Athena settings and data sources:

{ "region_name": "us-west-2", "output_location": "s3://your-athena-results-bucket/", "output_bucket": "your-athena-results-bucket", "database": "your-glue-database", "athena_workgroup": "your-athena-workgroup", "sources": { "wtk": { "bucket_name": "your-wtk-bucket", "athena_table_name": "wtk_table", "alt_athena_table_name": "", "capabilities": { "avg_types": ["all", "annual", "monthly", "hourly"] } }, "era5": { "bucket_name": "your-era5-bucket", "athena_table_name": "era5_table", "alt_athena_table_name": "", "capabilities": { "avg_types": ["all", "annual"] } } } }

Deploy

docker compose up --build

Access at http://localhost:5173

Clean up:

docker compose down --volumes --remove-orphans

Production Deployment

Build Individual Services

API:

cd windwatts-api && docker build -t windwatts-api:latest .

UI:

cd windwatts-ui && docker build -t windwatts-ui:latest .

AWS Configuration

RDS PostgreSQL: Configure DATABASE_URL for production database

Secrets Manager: Store WINDWATTS_DATA_CONFIG_SECRET_ARN with data source configuration

Environment Variables: Set production AWS credentials and endpoints

Load Balancer: Route traffic to containerized services

API runs on port 8000 using Gunicorn with Uvicorn workers.

RLiNREL · 2025-12-18T22:09:13Z

docs/03-backend.md

+
+Or using the wrapper:
+```bash
+python api.py -d


we don't have api.py any more.

RLiNREL · 2025-12-18T22:09:54Z

docs/02-quickstart.md

+### 1. Clone the Repository
+
+```shell
+git clone https://github.com/NREL/dw-tap-api.git


the repo name is changed to windwatts from dw-tap-api

RLiNREL · 2025-12-18T22:13:09Z

docs/02-quickstart.md

+Create a `.env` file in the root:
+
+```plaintext
+WINDWATTS_DATA_URL=https://windwatts-era5.s3.us-west-2.amazonaws.com/


@sameersk2k this data package is currently an internal package, is it? I am not sure whether we should include the actual URL here? but on the other hand, can others make this app running without the data package (should we release the .whl file publicly?

should probably release the data package honestly, since it is the bulk of the app (the data part), or, build this out as an sdk client that connects to NLR's deployed Athena/S3 data...it's a product decision

RLiNREL · 2025-12-18T22:17:39Z

docs/01-overview.md

+WindWatts is a monorepo consisting of:
+-   **Frontend**: React application (`windwatts-ui`)
+-   **Backend**: Python FastAPI application (`windwatts-api`)
+-   **Core Library**: Data processing logic (`dw_tap`)


dw_tap is not in use, part of the related functionalities is integrated to the backend utilities/core functions

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot · 2025-12-19T17:07:29Z

@shawnpetros I've opened a new pull request, #182, to work on those changes. Once the pull request is ready, I'll request review from you.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot · 2025-12-23T16:41:37Z

@shawnpetros I've opened a new pull request, #185, to work on those changes. Once the pull request is ready, I'll request review from you.

Copilot

Pull request overview

Copilot reviewed 47 out of 49 changed files in this pull request and generated 4 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

windwatts-api/alembic/env.py

Copilot · 2025-12-23T17:52:24Z

windwatts-api/Makefile

 .PHONY: openapi
 openapi:
 	. .venv/bin/activate; PYTHONPATH=. SKIP_DATA_INIT=1 python scripts/generate_openapi.py
+


The openapi target in the Makefile references scripts/generate_openapi.py, which was deleted in this PR. This will cause the make openapi command to fail. Either remove this target from the Makefile or restore the script.

Suggested change

.PHONY: openapi

openapi:

. .venv/bin/activate; PYTHONPATH=. SKIP_DATA_INIT=1 python scripts/generate_openapi.py

Copilot · 2025-12-23T17:52:24Z

docs/05-data-science.md

+### Setup
+
+```bash
+conda env create -f dw_tap/environment.yml
+conda activate dw_tap
+cd dw_tap
+python setup.py install
+```


The documentation references a dw_tap directory and package, but based on the PR description and context, this appears to be outdated naming. The codebase has been renamed to windwatts. Update references from dw_tap to the correct package/directory name.

Copilot · 2025-12-23T17:52:24Z

windwatts-api/app/power_curve/powercurve.py

+    # def plot(self):
+    #     fig = px.line(
+    #         y=self.powercurve_intrp(self.interp_x),
+    #         x=self.interp_x,
+    #         labels={"x": "Windspeed (m/s)", "y": "Power (kW)"},


This comment appears to contain commented-out code.

RLiNREL

The docs/05-deployment.md need to be updated, a draft was suggested.

RLiNREL · 2025-12-28T21:34:00Z

docs/02-quickstart.md

+- **UI**: http://localhost:5173/
+- **API**: http://localhost:8080
+
+## Option 2: Native Setup (Without Docker)


the native setup is not working anymore as the api.py for Flask app was removed

RLiNREL · 2025-12-28T22:01:23Z

docs/05-deployment.md

here is draft for the current set up using docker compose, the docker build command for Flask app is outdated.

# Deployment Guide ## Local Development Deployment Uses Docker Compose to run all services (API, UI, database, reverse proxy). ### Prerequisites 1. Docker and Docker Compose installed 2. AWS credentials with access to WindWatts data 3. Google Maps API key and Map ID ### Configuration **Root `.env`:** ```plaintext WINDWATTS_DATA_URL=<s3-url-for-windwatts-data-package> AWS_ACCESS_KEY_ID=<your-access-key> AWS_SECRET_ACCESS_KEY=<your-secret-key> AWS_SESSION_TOKEN=<your-session-token> WINDWATTS_DATA_CONFIG_SECRETS_ARN=<your-secrets-arn>

.env.development:

VITE_API_BASE_URL=http://windwatts-proxy:80 VITE_MAP_API_KEY=<your-map-api-key> VITE_MAP_ID=<your-map-id>

windwatts_data_config.json:

Configure AWS Athena settings and data sources:

{ "region_name": "us-west-2", "output_location": "s3://your-athena-results-bucket/", "output_bucket": "your-athena-results-bucket", "database": "your-glue-database", "athena_workgroup": "your-athena-workgroup", "sources": { "wtk": { "bucket_name": "your-wtk-bucket", "athena_table_name": "wtk_table", "alt_athena_table_name": "", "capabilities": { "avg_types": ["all", "annual", "monthly", "hourly"] } }, "era5": { "bucket_name": "your-era5-bucket", "athena_table_name": "era5_table", "alt_athena_table_name": "", "capabilities": { "avg_types": ["all", "annual"] } } } }

Deploy

docker compose up --build

Access at http://localhost:5173

Clean up:

docker compose down --volumes --remove-orphans

Production Deployment

Build Individual Services

API:

cd windwatts-api && docker build -t windwatts-api:latest .

UI:

cd windwatts-ui && docker build -t windwatts-ui:latest .

AWS Configuration

RDS PostgreSQL: Configure DATABASE_URL for production database

Secrets Manager: Store WINDWATTS_DATA_CONFIG_SECRET_ARN with data source configuration

Environment Variables: Set production AWS credentials and endpoints

Load Balancer: Route traffic to containerized services

API runs on port 8000 using Gunicorn with Uvicorn workers.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* 'feat/docs' of github.com:NREL/windwatts: Apply suggestion from @Copilot

shawnpetros · 2025-12-30T22:38:08Z

windwatts-api/scripts/generate_openapi.py

probably shouldn't delete this file...not sure why the linter did this

shawnpetros added 4 commits December 12, 2025 09:27

docs: overhaul documentation structure and content

4dcff55

chore: ignore cursor thoughts and plans

b19e0d1

fix gitignore

b88d2e7

fix gitignore

b931d18

Copilot AI review requested due to automatic review settings December 12, 2025 17:42

Copilot started reviewing on behalf of shawnpetros December 12, 2025 17:42 View session

Copilot AI reviewed Dec 12, 2025

View reviewed changes

RLiNREL reviewed Dec 18, 2025

View reviewed changes

Update docs/06-deployment.md

fc955bc

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI mentioned this pull request Dec 19, 2025

Add SHA256 checksum verification for windwatts_data wheel #182

Draft

shawnpetros and others added 7 commits December 19, 2025 09:07

Update docs/02-quickstart.md

91357a1

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/01-overview.md

9be18e8

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/02-quickstart.md

4332c88

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/05-data-science.md

ba0842a

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/05-data-science.md

a9c1c5d

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update README.md

d09bb4a

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Update docs/06-deployment.md

ca21994

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Copilot AI mentioned this pull request Dec 23, 2025

Update Docker image name to windwatts-api for consistency #185

Draft

shawnpetros added 2 commits December 23, 2025 09:42

docs updates

13e6301

added makefiles and tooling updates, linting

e6aff7b

shawnpetros requested review from RLiNREL and Copilot December 23, 2025 17:48

Copilot started reviewing on behalf of shawnpetros December 23, 2025 17:48 View session

Copilot AI reviewed Dec 23, 2025

View reviewed changes

shawnpetros added 4 commits December 23, 2025 09:54

language updates

5c4d0be

remove data science docs

46cf642

update the tooling to check and throw instead of fix in pre-commit

6d438ed

update script

8443d9e

RLiNREL reviewed Dec 28, 2025

View reviewed changes

shawnpetros and others added 4 commits December 29, 2025 08:37

Apply suggestion from @Copilot

f67836d

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

update the deployment doc

133b5f1

Merge branch 'feat/docs' of github.com:NREL/windwatts into feat/docs

7083df4

* 'feat/docs' of github.com:NREL/windwatts: Apply suggestion from @Copilot

updates to docs

756fe6a

shawnpetros commented Dec 30, 2025

View reviewed changes

RLiNREL previously approved these changes Dec 31, 2025

View reviewed changes

RLiNREL added 2 commits December 31, 2025 14:12

accept theirs for all unmerged

2dd33c2

make format api+ui

75895cd

RLiNREL dismissed their stale review via 75895cd December 31, 2025 21:26

RLiNREL merged commit 97ce051 into develop Dec 31, 2025

RLiNREL deleted the feat/docs branch December 31, 2025 21:27

RLiNREL mentioned this pull request Jan 2, 2026

License update #189

Closed

	.PHONY: openapi
	openapi:
	. .venv/bin/activate; PYTHONPATH=. SKIP_DATA_INIT=1 python scripts/generate_openapi.py

Feat/docs #176

Feat/docs #176

Uh oh!

Conversation

shawnpetros commented Dec 12, 2025 • edited by RLiNREL Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Key Changes

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Copilot AI Dec 12, 2025

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RLiNREL left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Deploy

Production Deployment

Build Individual Services

AWS Configuration

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI commented Dec 19, 2025

Uh oh!

Copilot AI commented Dec 23, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

RLiNREL left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

shawnpetros commented Dec 12, 2025 •

edited by RLiNREL

Loading