Creation of initial decision records.

docs/source/data/index.rst

@@ -1,5 +1,5 @@
 Data Overview
-================
+=============
 
 .. toctree::
    :maxdepth: 1

docs/source/decisions/0001-api-versioning.rst

@@ -0,0 +1,60 @@
+############################
+API Versioning and Reporting
+############################
+
+Summary
+=======
+
+API should provide a reason version and list/matrix of capabilities for a given instance of CDA
+
+Opinions
+========
+
+Opinion 1 Calendar versioning
+-----------------------------
+
+@MikeNeilson
+
+Summary: Calendar versioning is easier to support and automate with this API
+
+As the one whose has been making the "official" releases Semantic Version versioning has basically been useless.
+We have been making so many feature additions that if I was doing it right we'd never have a minor version change between releases.
+It's also caused me to, I think, release too slowly.
+
+While we can automate SemVer it is an additional step.
+
+With Calendar Versioning automation tools can just pick the current date when appropriately triggered,
+perhaps by merged into a particular branch.
+
+@adamkorynta 
+
+ Summary: Calendar versioning more directly aligns with consumers needs
+
+Deprecation, desupport, and removal notices usually involve timelines rather than predicting x number of major/minor releases into the future.
+
+Given the endpoints themselves are already versioned, following semver becomes obtuse with only a real use-case of structural changes such as supported JDK changes, etc. 
+
+Opinion 2 Users
+---------------
+
+Summary: It has been asked more than once that a version be provided
+
+Having a version allows client to better respond to what's available instead of failing in obtuse ways.
+
+
+Decision Status
+===============
+
+accepted
+
+1. Provide endpoint to retrieve current API version.
+2. Likely include capability list or matrix.
+
+References
+==========
+
+
+Related decisions
+=================
+
+- data-versioning

docs/source/decisions/0002-data-versioning.rst

@@ -0,0 +1,90 @@
+##################################
+Data Types use Calendar Versioning
+##################################
+
+Summary
+=======
+
+Instead of versioning the entire API, we version the data types if appropriate.
+This versioning only applies to simple non-breaking changes and it primarily presented as a hint in the returned
+data.
+
+Opinions
+========
+
+Opinion 1
+---------
+
+Summary: If a given end point can have additional, or modified, elements in the response, add a data version.
+
+
+@MikeNeilson
+
+By versioning the data, and using the Content-Type and Accept headers and the full features of MIME types we appropriately 
+separate the concern of "what data we are retrieving/storing" from the presentation of data.
+
+e.g /timeseries/Alder Springs.Temp-Air.Inst.15Minutes.0.GOES-raw?begin=PT0&end=PT-1D&units=C, granted with a reasonable
+ exception to the units, defines *what* we want.
+
+The header, Accept, informs the API what format, or formats, we are willing to accept the data in. 
+
+Opinion 2
+---------
+
+Summary: Remove data versioning in all endpoints.
+
+@adamkorynta 
+
+The accept/content-type headers have provided sufficient confusion to downstream clients and deviate from industry standards.
+I have yet to see an endpoint where this versioning solves the proposed problem of needing to different shapes of data (other than json vs xml).
+The concept was introduced to solve the straight-to-db queries, which did not have any OpenAPI documentation moving to in-app DTO's which now have documentation. When moving away from the straight-to-db queries, we needed to thoroughly expand queries parameters and make other backwards-incompatible changes unrelated to the data shape. Given that context in hindsight a path version would have worked better.
+
+The closest we've gotten to needing new shapes on the same endpoint is the data-entry date on TimeSeries, but this was
+more appropriately solved via query parameter and data arrays. I think if we had really wanted to be a stickler on the 
+"what format" we could have easily added another endpoint path instead. Even if/when we add text annotations,
+using a content type is obtuse given the lack of discoverability as we would then need `application/json`, 
+`application/json+data-entry+text-annotations`, `application/json+text-annotations`, `application/json+data-entry`
+which seems like just another type of bloat that is more hidden from clients.
+
+Decision Status
+---------------
+
+`rejected` - the descriptions in this proposal are awkward and it is not clear how to fix them. Additionally as we have
+decided to adopt path based versioning and we've made `application/json` or `application/xml` default to the latest desired 
+the requirement is now moot.
+
+Data, by content-types, are versioned. In the past there was some severe confusion on this part and it was treated as anything 
+new was "version=2" in the content-type. To allow this design but reduce confusion going forward
+
+1. The initial content-type of a data set *SHALL* be be the plain content-type and *SHOULD* include an additional expanded content-type
+3. *IF* is it not the first version of this data, additional information will be set in the content-type as 
+   appropriate to the to the data. (e.g. `application/json+<something>` or `application/json;<something>`)
+   1. It will be discussed and announced when it becomes the new default data, if that decision is made.
+5. Downstream systems *SHOULD* use the specific version regardless of when implemented, and this behavior should be well documented.
+6. If a given data set includes definitions of its shape within the type there should be sufficient documentation for downstream
+   developers to properly account for any changes over time. (See our TimeSeries type and discussions within #927).
+
+[comment:] <> (Status: request for comments | proposed | accepted | rejected | deprecated | superseded)
+
+References
+==========
+
+1. https://www.youtube.com/watch?v=jmoxGJ_sLgU
+2. https://newsletter.systemdesign.one/p/api-versioning
+3. https://www.speakeasy.com/api-design/versioning
+
+
+Notes
+=====
+
+The initial idea in CDA was that the first version of any data type was, we'll just stick with JSON for each of message,
+"application/json;version=1" with "application/json" being the alias to the latest format version. However, this was not
+correctly communicated and several brand new data transfer objects were created as ";version=2" under the impression that
+this was the version for the new system. Attempting to use a simple number of this has clearly caused confusion in general.
+
+We also failed to create the initial alias system which caused even more confusion when users attempted to test things 
+directly in a browser instead of the provided swagger-ui.
+
+
+Various practical concerns and common usage have also made doing this "pedantically correct" impossible to manage. The above
+should be a reasonable compromise.

docs/source/decisions/0003-searchability-and-catalogs.rst

@@ -0,0 +1,80 @@
+#################################
+Data should be readily searchable
+#################################
+
+Summary
+=======
+
+It's not just a good idea, it's technically the law, see reference 1 and 2. While CDA currently expose a fair amount 
+of information to search it's never entirely clear. We *MUST* adopt a standard method of searchability.
+
+Additional Information
+----------------------
+
+Catalog - a complete list of items, for examples of things that people can look at or buy [3]
+
+By having a well defined structure of information users can more easily discover what they are looking for. While the
+Swagger-UI, if used, presented all of the types of data that can be found. The `catalog` for each type should present
+a clear way to find the available data of each type.
+
+The catalog of each data set would include only metadata associated with each data type. For example a time series
+catalog would include support to discover primary timeseries names, aliases, extents, and the like but not actual time
+series data.
+
+However, for data that are primarily metadata, such as Locations, they would be one and the same.
+
+Opinions
+========
+
+Opinion 1
+---------
+
+Summary: Each data type should support it's own /catalog end point.
+
+@MikeNeilson
+
+The original CDA has the say, `/timeseries` end point provide a catalog if no data is set. I created a /catalog end point
+to attempt to consolidate search query parameters. For TimeSeries and Locations this works reasonably well since there
+is parity between the concepts.
+
+However, if we tried to add ratings into the mix, the list of query parameters grows, and it would rather difficult to 
+document which is for what or what changes for each.
+
+To make 'catalog' operations clear, we should create /catalog for each data type that provide for discoverability of that data.
+
+Opinion 2
+---------
+
+Summary: Each datatype should exit under a "/catalog"
+
+@MikeNeilson
+
+If it makes sense to group all catalogs under catalog, perhaps for grouping in the SWAGGER-UI, making each catalog it's own
+path under `/catalog` instead of the current path parameter is a better approach.
+
+We would maintain the grouping, but each catalog can have its appropriate search criteria. the `catalog/<datatype>` could 
+just redirect.
+
+Opinion 3
+---------
+
+Summary: Swagger-UI allows grouping under multiple tasks
+
+@Mike Neilson on behalf of @adamkorynta
+
+We can group the available catalogs into multiple Swagger-UI blocks while maintaining a `<datatype>/catalog`
+
+Decision Status
+===============
+
+proposed - requires additional discussion and likely some review after the first path based version is adopted.
+Document is left in proposed state to indicate additional ideas should be presented over time and as work is done.
+
+[comment:] <> (Status: request for comments | proposed | accepted | rejected | deprecated | superseded)
+
+References
+==========
+
+1. https://www.congress.gov/bill/115th-congress/house-bill/1770
+2. https://www.cio.gov/handbook/it-laws/ogda/
+3. https://www.oxfordlearnersdictionaries.com/us/definition/english/catalogue_1

docs/source/decisions/0004-versioning.rst

@@ -0,0 +1,84 @@
+########################
+CWMS Data Api Versioning
+########################
+
+
+Summary
+=======
+
+Maintaining backwards compatibility while improving future difficulty has proven sufficiently difficulty that change
+is required.
+
+The API as a whole will retain the calendar based versioning for formal releases.
+Data *SHOULD* be versioned, if appropriate/needed, with otherwise backwards compatible changes to query parameters.
+Endpoints will be placed under a new "api version" path parameter for backwards incompatible or confusing parameter changes.
+
+e.g.
+
+`https://host/cwms-data/locations`
+
+can become
+
+`https://host/cwms-data/v2/locations`
+
+As additional endpoints require such a change they should be added to an existing increased version. For each
+version all required verbs *SHALL* be implemented. e.g. the new version is a complete unit of operation.
+
+Example:
+
+given above and a v1 `timeseries` and yet another `locations` improvements
+
+.. code-block:: bash
+
+    # new time series becomes
+    cwms-data/v2/timeseries
+    # the new location becomes
+    cwms-data/v3/locations
+
+.. NOTE::
+
+    Or is that confusing and we should just allows add a new endpoint to the highest endpoint version?
+
+At Current time the "root" URL will be considered V1, and redirect to v1 urls.
+After X years the root URLs will redirect to the latest version.
+
+e.g.
+
+  .. code-block:: bash
+        # now
+        curl "https://cwms-data.usace.army/cwms-data/timeseries/Black Butte.Stor.Inst.~1Day.0.Calc-val?units=ft"
+        # will redirect to
+        curl "https://cwms-data.usace.army/cwms-data/v1/timeseries/Black Butte.Stor.Inst.~1Day.0.Calc-val?units=ft"
+        # after transition period, *IF* there is a new version
+        # will redirect to
+        curl "https://cwms-data.usace.army/cwms-data/v<next>/timeseries/Black Butte.Stor.Inst.~1Day.0.Calc-val?units=ft"
+        # if possible, query parameters can be updated on behalf of the user
+
+
+Opinions
+========
+
+Opinion 1
+---------
+
+Summary: Current scheme is not working
+
+Author MikeNeilson, on behalf of others
+
+We have failed to properly handle existing usages while attempting to improve the overall design of the api
+and have been breaking various downstream usages due to the confusion. Allowing the endpoints to be versioned allows
+an easier time keeping existing behavior while also allowing more drastic improvements in usages to happen.
+
+Decision Status
+===============
+
+Status: accepted
+
+[comment:] <> (Status: request for comments | proposed | accepted | rejected | deprecated | superseded)
+
+References
+==========
+
+1. https://www.youtube.com/watch?v=jmoxGJ_sLgU
+2. https://newsletter.systemdesign.one/p/api-versioning
+3. https://www.speakeasy.com/api-design/versioning

docs/source/decisions/adr-template.rst

@@ -0,0 +1,27 @@
+#####
+Title
+#####
+
+
+Summary
+=======
+
+Opinions
+========
+
+Opinion 1
+---------
+
+Summary: 
+
+Author
+
+descriptive text
+
+Decision Status
+===============
+
+[comment:] <> (Status: request for comments | proposed | accepted | rejected | deprecated | superseded)
+
+References
+==========

docs/source/decisions/index.rst

@@ -0,0 +1,23 @@
+################
+Design Decisions
+################
+
+
+Overview
+========
+
+
+Below are agreed upon decision choices regarding the usage of the API.
+Please note that certain decisions may be agreed upon before implementation. 
+Whether or not a particular choice is implemented will be marked for each decision record.
+
+Some decisions may also be a proposal and marked appropriately.
+
+.. toctree::
+    :maxdepth: 1
+    :caption: Decisions
+
+    Api Versioning <./0001-api-versioning.rst>
+    Data Versioning <./0002-data-versioning.rst> (rejected, remains for historical context.)
+    Catalogs and Search <./0003-searchability-and-catalogs.rst>
+    Versioning <./0004-versioning.rst>

docs/source/index.rst

@@ -38,8 +38,14 @@ Welcome to CWMS Data API documentation!
 
 
 .. toctree::
-    :maxdepth: 1
-    :caption: Alternative Topics
+   :maxdepth: 1
+   :caption: Alternative Topics
 
-    Alternative Topics <./alternative-topics/index.rst>
+   Alternative Topics <./alternative-topics/index.rst>
 
+.. toctree::
+   :maxdepth: 1
+   :caption: Design
+
+   Design <./introduction/design.rst>
+   Decision Records <./decisions/index.rst>