Skip to content

CDA BiWeeklies

Jump to bottom Edit New page
Charles Graham edited this page Dec 15, 2025 · 21 revisions

CDA BiWeekly Meetings

These page serves as notes and do-outs from the CWMS-Data-API biweekly meetings

Structure:

  • DO OUTS
  • INFO
  • Reported Bugs
  • Meetings

Do Outs

Documentation

  • List assumptions about what knowledge you are expecting the user to have as they look at this documentation. languages, etc
  • 101 Definitions for jargon throughout the pages. What is a "endpoint"
    • Perhaps a FAQ/jargon page
    • Talk about the various HTTP ranges 200s OK, 300s Redirect, 400s bad user input, 500s is bad server What IS a location category, and for all other endpoints a simple definition RIGHT at the top! CDA does not provide this, should it?

Bug Reports

View bug reports table
Original Discussion Priority Status Issue Other Times Brought Up Details
07/14/2025 ~ In Progress 1192
07/14/2025 ~ In Progress 1191
07/14/2025 ~ Issue Not Created • Locations Group - Requires office but the UI does not state this
curl -X 'GET' \  'https://cwms-data.usace.army.mil/cwms-data/location/group?category-office-id=MVP&location-office-id=MVP' \  -H 'accept: application/json'
From https://cwms-data.usace.army.mil/cwms-data/swagger-ui.html
07/14/2025 ~ Issue Not Created • Swagger spec for groups shows the description again in the category block within the group POST
○ Holdover from the swagger spec passing through the schema from a predefined block?
07/14/2025 ~ Issue Not Created Attempting to delete a location group that contains locations still results in a 500 generic error, return something more descriptive i.e. "delete locations first?"
07/28/2025 ~ Issue Not Created Offices, States, Countries, Units do not have a description of what the endpoint is or why it is(???)
07/28/2025 ~ Issue Not Created Parameter endpoint when specifying office verbiage:
"If this field is not specified, the session user's default office will be used."
From https://cwms-data.usace.army.mil/cwms-data/swagger-ui.html
07/28/2025 ~ Issue Not Created TimeZone endpoint for JSON returns a list of key/value pairs. Without additional response key/values it might be OK to just return the timezone.
Theresa noticed If you explicitly provide json in the format query parameter it will return the result but with an invalid JSON format.
curl -X 'GET' \  'https://cwms-data.usace.army.mil/cwms-data/timezones?format=json' \  -H 'accept: application/json;version=2'
From https://cwms-data.usace.army.mil/cwms-data/swagger-ui.html
09/09/2025 Issue Not Created Attempting recent with group/category returns empty list. User error?
09/09/2025 ??? No Data in the Profile responses for any office!
09/09/2025 Reported Already 1101 Custom datum missing from timeseries endpoint?
09/09/2025 ??? Querying with trim disabled will not return the value that returns if you enable trim?
curl -X 'GET' \  'T7:8243/swt-data/timeseries?name=KEYS.Elev.Inst.1Hour.0.Ccp-Rev&office=SWT&begin=2013-07-01T01%3A00%3A00Z&end=2014-07-01T01%3A00%3A00Z&trim=false' \  -H 'accept: application/json;version=2'
From https://T7:8243/swt-data/swagger-ui.html
Possibly related to the page-size? What effect does that have on trim false vs true
09/09/2025 1257 CSV on TS endpoint returning:
"One or more provided values exceeds the maximum length for the parameter."
From https://T7:8243/swt-data/swagger-ui.html
09/22/2025 Issue Not Created Attempting to POST invalid JSON payload with TimeSeries endpoint (or any other JSON payloads) gives a full error output. Reduce error output to just column number/line number and suggest to user it's invalid JSON?
09/22/2025 Issue Not Created TimeSeries Identifier endpoint has both name and timeseries-id, possibly just timeseries-id?
09/22/2025 Issue Not Created Attempting to storemultiple gives a 500, server does recognize that it is multiple values input in error log. Creates the ID but stores no value, is this expected behavior?
10/20/2025 Issue Not Created Caused by: com.fasterxml.jackson.databind.exc.InvalidFormatException: Cannot deserialize value of type java.time.Instant from String "2025-10-20T00:00:00": Failed to deserialize java.time.Instant: (java.time.format.DateTimeParseException) Text '2025-10-20T00:00:00' could not be parsed at index 19 : "date-time": "2025-10-20T00:00:00",'
10/20/2025 Issue Not Created Both text-timeseries and binary-timeseries include blob-id but should not required it (depreciated)
12/01/2025 #1184 Cannot delete TS Cat/Group if it has TSIDs assigned

Meetings

06/16/2025 - Introduction / Auth Endpoints

  • Introductions

  • Purpose

    • USACE collab on our future access to the database -> JDBC is going away

  • Versions of CDA

    • Check your version if you have issues: install here: https://github.com/USACE/cwms-data-api/wiki/Installing-into-Tomcat#for-the-usace-t7-systems-follow-these-steps
    • Neilson: The data entry date change was merged in, should be available from water.dev.cwbi.us/cwms-data
      - it's not availalbe in cwbi-test or -prod yet or a release.
    • Check URL you are using, national may not match your T7 version, if you want the latest run the above script
    • You can see what version of YOUR CDA instance on your T7 is by going to the swagger page, should be a gray radial box with the version in it. If you do not have this, update to the latest using the link above!

  • Documentation Website.

    • https://cwms-data-api.readthedocs.io/latest/
    • How to contribute?
      - In this meeting - we talk and make issues on Github
      - You think of something, make an issue here: https://github.com/usace/cwms-data-api/issues
      - Submit a pull request, if you would like to directly add to the docs - we can have a meeting to discuss how to do this.
      - It would go through a Peer Review process (PR) and reviewed by all of the dev/team under GitHub.
      - Possible to make a reviewer group that ONLY gets documentation PR and not code PR
      - "Code owners concept"
      - Also could do a group session one or more times and actually make the PR/doc changes ourselves
    • Intent: - Help stakeholders not only with development, but understanding CWMS and what CDA actually provides
      - Open hydrologic / hydrology discussions related to what CWMS has created since its inception
      - Internally both non hydrology based and other fields of work can benefit from this - users not "in the know"

  • Plans moving forward

    • Endpoints
      - Drive discussion for that given day -> Walk through these and decide if they need more documentation/are otherwise "broken" (for our needs)
    • Started walking through endpoints
      - https://water.dev.cwbi.us/cwms-data/
    • Went through Auth and / endpoints here:
      Default, Auth Endpoints
06/30/2025 - Location Categories/Groups

  • Questions or Concerns

    • No Concerns/comments

  • Location Categories

    • Overview / Go through endpoints
    • NOTE: Categories have groups (groups endpoint later)
    • GET
      • getOne -> Provides the metadata (description) of a provided category id
      • getMany -> If you type an office you will not see the same results CWMSVue shows as CWMSVue is showing CWMS office-id as well (at least for the RMI connection)
    • POST
    • DELETE

  • Location Groups

  • Documentation

    • Update/Add any?
      • NWP: might want to list assumptions about what knowledge you are expecting the user to have as they look at this documentation. languages, etc
      • 101 Definitions for jargon throughout the pages. What is a "endpoint" - perhaps a FAQ/jargon page
        • Talk about the various HTTP ranges 200s OK, 300s Redirect, 400s bad user input, 500s is bad server
      • What IS a location category, and for all other endpoints a simple definition RIGHT at the top! CDA does not provide this, should it?
    • Does public use these endpoints?
    • Do we have these on our public sites?

  • Query /locations with only office set

    • CSV not supported/other types? Remove these from UI
07/14/2025 - Locations
  • Examples ->
  • cwms-python
  • In the swagger docs themselves include base examples
  • What are location IDs? What are names?
  • Gold stars
  • Wont let you create location without lat/lon/timezone
  • Bug
  • Get a 500 if you don't specify something on /locations (no id) but it also says not required for all parameters cwms.cda.api.LocationController.getAll -1177377251864346337: failed to process request java.time.DateTimeException: Invalid ID for region-based ZoneId, invalid format: Unknown or Not Applicable Timezone not set for any given location that results in the query would cause this? - SPL (Paul/Jason) ran into this querying the locations endpoint
  • In the swagger UI it requires office, but the verbiage is the preferred
  • Locations-id patch in swagger defaults to xml request body
  • Confirm all endpoints respond accordingly if nothing is provided when there are no required params
  • PATCH endpoint does not respect units
  • Swagger UI expects a json response but a plaintext string is returned when the location is updated or created (should be JSON)
    • Happens for POST/PATCH/DELETE (check other endpoints)
  • Bypass UI and attempt to query endpoint without office results in null office and not found location If this field is not specified, matching location level information from all offices shall be returned.
07/28/2025 - Counties, States, Offices, Parameters

Issues in the Field

  • LRH -> CDA issue with CAVI - Script was targeting CDA-Test, but now CDA-Prod is working. CAVI required a specific version of CDA(?)

Side Convos

  • Unit vs Units

  • No agreement on this

  • Avoid tab and csv format - possibly going away(?)

  • States:

    • These are referenced to the locations and their counties within the database
    • NOTE: we should update the swagger spec to mention this possibly?
      • Mention provinces/islands etc
    • Counties:
      • Also related to states
        • Counties/States:
        • Ideally enable dropdown use to select these for locations because you cannot make custom values(??)
        • Cached by browser by default using cache control header = 3600 seconds
      • Offices
        • Gold star, returns if the office doesn't exist
        • GET - getOne (Single endpoint works OK) - Returns single office
        • GET - getAll (Catalog of offices)
        • Uses the extents for the has-data parameter (fast)
      • Parameters
        • Office id specified to allow or offices to make sub parameters
        • Sub Parameter vs base parameter
08/11/2025 - Forecast + Rate + Questions
  • Opening
    • Question about whether there was a carry-over plan from last meeting.
    • John suggested an approach; agreed it might work.
    • Issue raised with not being able to open the OneNote document.
  • Ratings Endpoint
    • Confirmed the ratings endpoint exists.
    • Not yet available in CWBI Test, but present in CWBI Dev (latest CDA version).
    • Should not fail completely on time series; ratings will be included.
    • Some attendees have tested it; more testing encouraged.
  • Environment / Versioning
    • Local T7 CDA may need a refresh/update to use Rate (Schema Update)
    • Multiple versions can run; requires extra characters in the identifier.
    • Convention: start with OfficeID-Data, then append dash + custom suffix.
  • General Updates
    • Problems with driving database updates mentioned.
    • Discussion of whether public-facing CDA is ready for partners to access.
    • Legacy scripts currently used for pulling data; desire to transition.
  • Action Items / Next Steps
    • Refresh local T7 CDA environment.
    • Continue broader testing of ratings endpoint.
    • Evaluate readiness of public CDA for external partner access.
    • Track database update issues for resolution. These notes were generated using AI + the Teams Transcript. They were reviewed by Charles Graham, SWT for errors
08/25/2025 - Levels

Notes from Meeting: Cascade delete on level-id? Mitch thinks it could be for multiple dated versions, Charles agrees

  • Versions vs Effective Date: A level effective date does NOT overlap but a version (timeseries) can! -NWP

Gold Stars Won't let me specify multiple unit types!

{"message": "Mutually exclusive fields used.",
  "incidentIdentifier": "user input error",
  "details": {
    "Use only one of": [
      "Only one of the following can be defined for location levels: constant-value, seasonal-values, seasonable-time-series-id"
    ]
  }
}
  • Bugs

    • Trying to patch the level endpoint using the same level ID returns a 404? Possible user error? Caused by: Error : 20048, Position : 0, Sql = BEGIN "CWMS_20"."CWMS_LEVEL"."RENAME_LOCATION_LEVEL" (:1 , :2 , :3 ) ; END;, OriginalSql = { call "CWMS_20"."CWMS_LEVEL"."RENAME_LOCATION_LEVEL" (?, ?, ?) }, Error Msg = ORA-20048: NO_WRITE_PRIVILEGE: User doesnt have write privilege

    • Trying to delete a level in CDA Swagger returns error:

      • Caused by: Error : 20048, Position : 0, Sql = BEGIN "CWMS_20"."CWMS_LEVEL"."DELETE_LOCATION_LEVEL" (:1 , :2 ) ; END;, OriginalSql = { call "CWMS_20"."CWMS_LEVEL"."DELETE_LOCATION_LEVEL" (?, ?) }, Error Msg = ORA-20048: NO_WRITE_PRIVILEGE: User doesnt have write privileges
      • Specified levels patch/delete returns a 204, no content, but when you create that is a 201
      • Probably normal response for a POST is 201?
09/08/2025 - TimeSeries - D1
  • TimeSeries Endpoints
  • Uri "format=" and header are interfering with each other where format= is the legacy and header: will replace
09/22/2025 - TimeSeries - D2
  • Moving these notes to a place accessible (this wiki!) outside the Corps network (Sharepoint is blocked)
  • /timeseries/filtering
  • /timeseries/
  • /timeseries : POST
    • Does not specify default for replace rule or override protection (Swagger UI)
  • /timeseries/identifier-descriptor
    • Delete entire timeseries ID / DELETE
10/06/2025 - Standard Text / Text Timeseries

  • Std Text

    • GET requires AUTH? Worked for some without auth, others reported cwms-data / local T7 required auth before a GET request would work. Confirmed with other person that it does work without auth. May have been an issue with which CDA instance we targeted/login state.
    • POST Works No problem!
    • GET by mask - fails, the mask would seemingly take anything <todo: ticket>
    • DELETE - Returns a 500 if I try to delete an ID that does not exist
    • NOTE: What is the purpose of the various methods of DELETE, trying one over the other appears to have the same results:
      • Specifies the delete method used : Available values : DELETE_ALL, DELETE_KEY, DELETE_DATA

  • Text TimeSeries

    • Do these show up in the catalog endpoint? NAB for Cowanesque.Text.Inst.0.0.Gate Change Log does not but has data on the text-ts endpoint
    • Note, text-timeseries uses CLOB currently. But in the future will not be using CLOB table. (S3 eventually?)
    • Missing date example for /timeseries/text/{name}/value
    • If a value is large, over 64 KB, it returns the URL of the value endpoint above. This works, but users end up needing to learn the clob-id and TS code to query the value endpoint. PLANNED to be fixed (ticket made?)
    • TODO: needs better documentation all around on why/how you use these endpoints (future text-binary convo)
    • Delete: Uncertain what a text-mask is, it is required. Do we keep this?
      • This endpoint appears to delete JUST the data, using TS Identifier will delete the actual ID.
10/20/2025 - Binary TimeSeries (v2025.09.02-testa)
  • Binary TS
    • Document (for each endpoint) that it requires a base64 encoded string (cwms-python handles this)
    • Better error handling for date "date-time", missing timezone. Adding 'Z' at the end of the time fixed issue.
    • /timeseries/binary get, post, delete endpoints appear to work as expected.
    • Attempted updating quality flag in patch endpoint. Was not successful. (Cleared cache). The 'patch' response appeared successful but the quality flag did not appear updated on subsequent retrieval.
    • Discussion as to whether patch should require entire payload or just the fields to update. (Currently requires entire payload)
    • Issues will be tested with latest code to verify they persist before logging issues.
11/17/2025 - /timeseries/identifier && /timeseries/category

  • Talked about Dev -> Test -> Prod | T7 is PROD way to write data until a district is FULLY migrated to cloud (no more T7). Gain access to Dev/Test - Email Eric N.

  • Time Series Categories

    • Dev might only shows CWMS as not all offices / categories are entered

  • /timeseries/category - works as expected

    • No pagination?
    • /timeseries/category/{category-id} - this returns what you have to provide it, room for improvement? Not a useful endpoint if it returns what you provide? Regex did not work (ignoring swagger description)
    • Attempting to DELETE different office ID category ID yields a 500. Would expect a 403/forbidden.
    • Did not check logs: "incidentIdentifier": "0b263a6b-2305-46c2-bd88-6e08dd06a008", 
curl -X 'DELETE' \
  'https://water.dev.cwbi.us/cwms-data/timeseries/category/Default?office=CWMS&_cb=1763400651300' \
  -H 'accept: */*' \
  -H 'Cache-Control: no-cache, no-store, max-age=0' \
  -H 'Pragma: no-cache'
  • /timeseries/identifier-descriptor
    • Not specifying an office returns blank list for identifier. PROD (2025.09.02) and DEV (2025.11.17-develop)
    • Total not included in response despite being in the swagger spec(?) || DEV (2025.11.17-develop)
    • Attempting to query 10,000 on the identifier does not respond.. may eventually?
    • PATCH -> name and office-id exist, should they? They are post listed in path in the UI
    • If you have snap options in PATCH, they should exist in the schema default for POST. Docs in TimeSeries on Read the Docs?
    • no questions with DELETE
12/01/2025 - Documentation Update Tested with: `2025.12.01-develop`
  • Speed of these meetings, could be faster. Open to feedback if maybe weekly?
  • Read the docs page is live with TS changes: https://cwms-data-api.readthedocs.io/latest/
  • Covered 500 errors for local CDA instances (T7) - Usually account locked or invalid password ON COOP box
  • [GET] /timeseries/group
    • office parameter is for the individual timeseries office-id aka "assigned", the docs mention this. But we missed it at a glance
    • Documentation - An example highlighting the difference on when to filter office-id vs category-office-id
    • Docs => Talk about when to use include-assigned to know it might be good for initial TTL
  • [GET] /timeseries/group/{group-id}
    • need to return assigned timeseries to the group, right now they are emptY
    • the swagger doc spec is camelCase should be kebab case (hyphen)
  • [POST] /timeseries/group
    • If you create a group and the TSID does not exist, returns an error but still makes the group. Should it fail to create group? If so, document
    • Gold star, can't make TS groups with assigned TS that don't have a TS for the incorrect office
    • ISSUE: Delete - no option to cascade delete TSIDS from category, Option exists in category group endpoint to delete group. But not TSIDS, gets same error. If TS group has assigned category TSIDs, error returns as 500 but logs show: Error Msg = ORA-20998: ERROR: Cannot delete time series group monthly/web-reports because it is not empty.
      • Attempted to POST and PATCH empty TS array, and did not overwrite TS category TSIDS
    • Issue: Creates assigned timeseries where the group/category id are one office, but the assigned TS is another office. Errors saying loc-id does not exist, however, appears to still create the group. I did go and delete the group entirely and start fresh.

Payload

{
  "office-id": "SWT",
  "id": "web-reports",
  "time-series-category": {
    "office-id": "SWT",
    "id": "monthly",
    "description": "Reports for project timeseries for public data"
  },
  "description": "Reports on public and internal web",
  "assigned-time-series": [
    {
      "office-id": "LRL",
      "timeseries-id": "ACTI3.Flow.Inst.6Hours.0.OHRFC-0hrQPF",
      "attribute": 0
    }
  ]
}
  • NOTES:
    • Docs for how to auth to swagger? Do those exist?
curl -X 'GET' \
  'https://water.dev.cwbi.us/cwms-data/timeseries/group/SHEF%20Data%20Acquisition?office=CWMS&category-office-id=CWMS&group-office-id=CWMS&category-id=Data%20Acquisition' \
  -H 'accept: application/json'

curl -X 'GET' \
  'https://water.dev.cwbi.us/cwms-data/timeseries/group?office=SWT' \
  -H 'accept: application/json'
12/15/2025 - Ratings Tested with: 2025.12.15-develop
  • /ratings/rate-values/{office}/{rating-id}
    • [Needs Info] On DEV (2025/12/15) we got a 400 testing with LRH
    • [Needs Info] On TEST (2025.10.22-testa) we got a 500
    • Attempting Multiparam query unsure on format, need examples -> KEYS.Opening-Spillway_Gates,Elev;Flow-Spillway_Gates.Standard.Production
    • Get a generic "Invalid CWMS DTO Provided" error for various errors. For examle # values count does not match # of value-times
{
  "message": "Invalid CWMS DTO provided",
  "incidentIdentifier": "user input error",
  "details": {}
}

  • /ratings/rate-ts/{office}/{rating-id}

    • Give one TSID(or more if multi param) and return back a list of rated values, slows down if you query MANY (rate limited?)
    • Need docs for trim/start(end)-inclusive/previous/next, prev/next are boolean flags. If you specify either it will grab nearest value before and after time windows respectively

  • /ratings/reverse-rate-values/{office}/{rating-id}

    • Worked as expected, reverse input value/units to go the other direction using the same curve

  • /ratings/reverse-rate-ts/{office}/{rating-id}

    • As expected, note to give the output param TS as the input

  • User Notes about ratings:

    • Make notation if all input values are millis or seconds, if you enter seconds it will do the beginning of the records(?)
      • Add to Ratings in read the docs / consider enforcing millis on all data inputs, add to CONTRIBUTING?
    • Comment about page sizes, no pagination on ratings
Add a custom footer
Add a custom sidebar

Clone this wiki locally