Release Best Practices Guide #149

riverma · 2024-03-26T03:44:17Z

Purpose

A guide to help make release-making as consistent and automated as possible
PLEASE REVIEW LIVE VERSION HERE: https://github.com/NASA-AMMOS/slim/tree/issue-97/docs/guides/documentation/releases
- Help provide feedback in this PR by testing it out on your project!

Proposed Changes

[ADD] A draft guide text
[ADD] A template release.yml file to help make release notes look good and standardized with categories
[ADD] A python script to help aggregate release notes from multiple GitHub repositories into a single file

Issues

[New Process Improvement Need]: Consolidated Release Notes #97

Testing

Guide successfully rendered in SLIM docusaurus view (locally)
See template release.yml in action for the SLIM project here: https://github.com/riverma/slim/releases/tag/v1.4.0

Automation release notes aggregation script tested successfully on all SLIM repositories with the following output:

$ python gh_aggregate_release_notes.py gh_aggregate_release_notes_config.yml 
Downloading release notes: 100%|█████████████████████████████████████████████████████████████| 2/2 [00:00<00:00,  3.47it/s]
# slim-starterkit-python

## What's Changed
* Release Update of Python Starter Kit with Scanning by @ingyhere in https://github.com/NASA-AMMOS/slim-starterkit-python/pull/14
* NASA-AMMOS/slim#89: Functional GitHub Actions-based secrets detection… by @ingyhere in https://github.com/NASA-AMMOS/slim-starterkit-python/pull/10
* NASA-AMMOS/slim#25: Functional GitHub Actions-based SCRUB (CodeQL) analysis... by @ingyhere in https://github.com/NASA-AMMOS/slim-starterkit-python/pull/11
* NASA-AMMOS/slim#110: Functional Pylint static code analysis. ... by @ingyhere in https://github.com/NASA-AMMOS/slim-starterkit-python/pull/12
* Add Governance documentation by @jpl-jengelke in https://github.com/NASA-AMMOS/slim-starterkit-python/pull/9

**Full Changelog**: https://github.com/NASA-AMMOS/slim-starterkit-python/compare/1.0.0...1.0.1

# slim-starterkit

## What's Changed
* Remove duplicate text. Standardize text for link insertion. by @jpl-jengelke in https://github.com/NASA-AMMOS/slim-starterkit/pull/2

## New Contributors
* @jpl-jengelke made their first contribution in https://github.com/NASA-AMMOS/slim-starterkit/pull/2

**Full Changelog**: https://github.com/NASA-AMMOS/slim-starterkit/commits/v1.0.0

riverma · 2024-03-26T03:57:08Z

@mike-gangl - would this help Unity make releases across many repositories? Let me know what you think.

Standardized release template for SLIM as described in #149

docs/guides/documentation/releases/README.md

nutjob4life

Token handling is probably the serious issue but see also my other comments which you can take or leave 😀

docs/guides/documentation/releases/README.md

nutjob4life · 2024-03-26T16:37:49Z

docs/guides/documentation/releases/README.md

+   2.1 Download our script 
+   - Copy/download our script to your local machine
+      ```
+      curl -o gh_aggregate_release_notes.py https://raw.githubusercontent.com/NASA-AMMOS/slim/issue-97/docs/guides/documentation/releases/gh_aggregate_release_notes.py


Personal taste: in documentation like this, I always prefer the --long format command-line arguments as they feel more "documentary" 😁 , so -o → --output

Please remember to switch that link to the main branch before releasing. I only mention it here because it is something that often slips through the cracks.

Thanks @nutjob4life - fixed in latest commit.

@jpl-jengelke - will keep this comment unresolved so I don't forget!

nutjob4life · 2024-03-26T16:38:19Z

docs/guides/documentation/releases/README.md

+
+   2.2 Create a configuration file with your project's release information. Save the file with extension `.yml` A sample is provided below:
+      ```
+      github_token: <INSERT YOUR GH PERSONAL TOKEN>


Could use a brief hint here on how to get a token—maybe down on the FAQ?

In retrospect, I think this is awfully risky. Your token could afford a lot of permissions unless it's carefully scoped—and putting it into a file makes it too easy to check into version control (yay detect-secrets!) Maybe it should be an environment variable?

My comment is to link to the GH Actions documentation directly -- I wouldn't put "" here, but instead I would make it explicit with ${{ secrets.GITHUB_TOKEN }}. Then in 2.2, something like, "Create a release notes template (.yml) that's authorized to publish your project."

I think that's OK because "secrets.GITHUB_TOKEN" is mnemonically explicit.

@nutjob4life - I was debating between an environment variable and a file. I felt that a file could be kept secure on a system with permission control, whereas the environment variable approach keeps a credential on the command-line history. Though I see the risk of committing this file and your point! Are these our only two choices, I'm curious about best practices here.

@jpl-jengelke - agreed to linking to GH directly and letting their docs handle the UI specifics.

docs/guides/documentation/releases/README.md

nutjob4life · 2024-03-26T16:39:53Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

@@ -0,0 +1,99 @@
+import requests


The script should probably include some comments at the top about dependencies. For example, it does import requests so needs a Python installation (or virtual environment) with Requests installed. Bonus points for mentioning a version number of requests too 👍 (same for tqdm, yaml).

Echoes my comment above... Also, Python 3+.

Also may want to add a little try-except error catching to explain to a casual user what's wrong. What if the template itself is munged up? That can be hard to catch.

Some minimal logging (even just print statements) may be helpful, too.

nutjob4life · 2024-03-26T16:53:08Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

+        if response.status_code == 200:
+            return response.json().get("body", "")
+        else:
+            print(f"Error fetching release notes for {url}: {response.status_code}. Retrying...")


Maybe move the retry_num += 1 up and add an if retry_num < max_retries to condition the sleep? The "edge" case is that GitHub gives non-200 every single time, but we end up doing a final sleep for 2 × 2⁴ + [0.0..1.0] seconds—up to 33 seconds—to then just abort without a final attempt.

In other words, change from attempt…sleep…attempt…sleep…attempt…sleep…attempt…sleep…abort → attempt…sleep…attempt…sleep…attempt…sleep…attempt…abort 🤷

Not a huge deal but I've been known to be rather pedantic 😂

Let me look into this thanks @nutjob4life

nutjob4life · 2024-03-26T16:57:38Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

+        print("Failed to read configuration.")
+        return
+
+    github_token = config.get("github_token")


See comment above about putting the token into the .yml (or .yaml where I come from 😁) file.

jpl-jengelke

Maybe some changes, but specifically with the token stuff, would be good.

jpl-jengelke · 2024-03-26T17:55:57Z

docs/guides/documentation/releases/README.md

+   2.1 Download our script 
+   - Copy/download our script to your local machine
+      ```
+      curl -o gh_aggregate_release_notes.py https://raw.githubusercontent.com/NASA-AMMOS/slim/issue-97/docs/guides/documentation/releases/gh_aggregate_release_notes.py


Please remember to switch that link to the main branch before releasing. I only mention it here because it is something that often slips through the cracks.

jpl-jengelke · 2024-03-26T18:08:57Z

docs/guides/documentation/releases/README.md

+
+   2.2 Create a configuration file with your project's release information. Save the file with extension `.yml` A sample is provided below:
+      ```
+      github_token: <INSERT YOUR GH PERSONAL TOKEN>


My comment is to link to the GH Actions documentation directly -- I wouldn't put "" here, but instead I would make it explicit with ${{ secrets.GITHUB_TOKEN }}. Then in 2.2, something like, "Create a release notes template (.yml) that's authorized to publish your project."

jpl-jengelke · 2024-03-26T18:09:53Z

docs/guides/documentation/releases/README.md

+
+   2.2 Create a configuration file with your project's release information. Save the file with extension `.yml` A sample is provided below:
+      ```
+      github_token: <INSERT YOUR GH PERSONAL TOKEN>


I think that's OK because "secrets.GITHUB_TOKEN" is mnemonically explicit.

jpl-jengelke · 2024-03-26T18:17:43Z

docs/guides/documentation/releases/README.md

+   2.3 Run the script to generate aggregated release notes
+      ```
+      $ python gh_aggregate_release_notes.py your_configuration_file.yml
+      ```


OK, I like the concept of building release notes based off the GH API and the script looks light enough. When I first read about this I though of using GH's built-in facilities to just copy it. Could it all be done in an action file without a separate Python component? What if the actual release notes stored in the 'releases' section of the repository are used or the macros that create them there are reused? I think if GH products are reused then GH handles maintenance, the notes are uniform and the maintenance footprint is reduced. (I can see Python versions and module releases being pesky as time goes forward, plus shouldn't there be a dependencies file (requirements.txt)?

jpl-jengelke · 2024-03-26T18:20:21Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

@@ -0,0 +1,99 @@
+import requests


Echoes my comment above... Also, Python 3+.

jpl-jengelke · 2024-03-26T18:23:10Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

+    }
+
+    max_retries = 5
+    retry_num = 0


You could use Tenacity here to automate retries over the function. It would also handle the manifest other potential exceptions that could occur.

jpl-jengelke · 2024-03-26T18:25:08Z

docs/guides/documentation/releases/gh_aggregate_release_notes.py

@@ -0,0 +1,99 @@
+import requests


Also may want to add a little try-except error catching to explain to a casual user what's wrong. What if the template itself is munged up? That can be hard to catch.

Some minimal logging (even just print statements) may be helpful, too.

jpl-jengelke · 2024-03-26T18:26:25Z

docs/guides/documentation/releases/release.yml

Having a YAML-based template, which is very pedantic on format, may be a source of issues for casual users who aren't very pedantic with spacing, etc.

pogi7 · 2024-03-27T00:09:15Z

docs/guides/documentation/releases/README.md

+1. **Customize Release Notes**:
+   - Create a `.github/release.yml` file in your repository.
+   - Download our [GitHub release notes template](release.yml) and place the contents into your `.github/release.yml` file.
+   - Customize our recommended template as a baseline and to configure how release notes are generated based on your project's issue labels.


@riverma Adding a picture(s) of how to this step generates release notes would be helpful for new users wanting to adopt SLIM.

Great suggestion @pogi7 - will do here 👍

riverma · 2024-04-03T18:19:37Z

Received a use case from the OPERA project regarding the following question: how do we safeguard that a super release will include the right versioned repositories?

In other words, can this best practice guide PR help ensure that a particular build or release of many repositories doesn't lead to a situation where the wrong version of a particular repository's code is mixed up with another repository during a super release?

CC @philipjyoon @hhlee445

mike-gangl · 2024-04-12T15:22:02Z

@mike-gangl - would this help Unity make releases across many repositories? Let me know what you think.

So in terms of unity, we keep a changelog which is where most release notes comes from; thought this can still be very useful- there is a bit of a difference in what goes into a changelog and what we would describe at a system level release (which, in my opinion, is slightly different than these super releases. I can see "super releases" on a service area basis. The System releases, however, would be manually currated and de-jargoned for a slightly different audience.

stirlingalgermissen · 2024-07-11T18:56:04Z

Recommend using AI/ML to process code commits in addition to commit and PR comments for generating release notes.

There is significant benefit if AI/ML can extract release note information from code from that has not already been captured as a developer comment.

riverma · 2024-07-16T16:57:57Z

Great idea! Will consider a prompt scenario to support this. Rishi

…

________________________________ From: stirlingalgermissen ***@***.***> Sent: Thursday, July 11, 2024 11:56:26 AM To: NASA-AMMOS/slim ***@***.***> Cc: Verma, Rishi (US 398B) ***@***.***>; Mention ***@***.***> Subject: [EXTERNAL] Re: [NASA-AMMOS/slim] Release Best Practices Guide (PR #149) Recommend using AI/ML to process code commits in addition to commit and PR comments for generating release notes. There is significant benefit if AI/ML can extract release note information from code from that has not already been captured as a developer comment. — Reply to this email directly, view it on GitHub<https://urldefense.us/v3/__https://github.com/NASA-AMMOS/slim/pull/149*issuecomment-2223672415__;Iw!!PvBDto6Hs4WbVuu7!OV9IQiv9KLKLbgpTfGg0oR_EHKxA5kY3SCIKx-TznvhjJwQ8kQ9pfBgfll6M8-MlZiGZDd36yHXoAYqwVWtp2yYNjvyOuA$>, or unsubscribe<https://urldefense.us/v3/__https://github.com/notifications/unsubscribe-auth/AAX36LWWZUZ6TI2YT4LGQYTZL3INVAVCNFSM6AAAAABFIE2S7SVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMRTGY3TENBRGU__;!!PvBDto6Hs4WbVuu7!OV9IQiv9KLKLbgpTfGg0oR_EHKxA5kY3SCIKx-TznvhjJwQ8kQ9pfBgfll6M8-MlZiGZDd36yHXoAYqwVWtp2ybCMkpDDA$>. You are receiving this because you were mentioned.Message ID: ***@***.***>

sonarqubecloud · 2025-01-28T22:25:32Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

riverma · 2025-05-13T05:06:28Z

Taking a new look at this PR, I think we should consider augmenting it with a few additional features to address some important release management questions that have come up recently from #188 and NASA-AMMOS/slim-cli#31:

Third-party repository tagging: How do we tag development releases in third-party repos like PyPI? We should define a consistent approach.
Release ownership: Who owns the release process, particularly for publishing to external repositories like PyPI? Clear ownership will help ensure smooth releases.
Version number management: How do we systematically manage version numbers? A structured approach would be beneficial.

Potential solutions to consider:

The Conventional Commits specification for standardized commit messages
A commit linter like commitlint to enforce the convention

These two items would serve as prerequisites for implementing an automated release management solution like:

Both options can automatically update the CHANGELOG, create releases, and publish to third-party repositories based on conventional commits.

cc: @anilnatha @rtapella @yunks128 for awareness and input

yunks128 · 2025-05-14T03:12:03Z

@riverma referring to this pull request (NASA-AMMOS/slim-cli#31), All three of your concerns—third-party repository tagging, release ownership, and version number management—can be effectively addressed by adopting semantic-release for automated versioning and release workflows based on commit history. What do you think?

Taking a new look at this PR, I think we should consider augmenting it with a few additional features to address some important release management questions that have come up recently from #188 and NASA-AMMOS/slim-cli#31:

Third-party repository tagging: How do we tag development releases in third-party repos like PyPI? We should define a consistent approach.

Release ownership: Who owns the release process, particularly for publishing to external repositories like PyPI? Clear ownership will help ensure smooth releases.

Version number management: How do we systematically manage version numbers? A structured approach would be beneficial.

Potential solutions to consider:

The Conventional Commits specification for standardized commit messages

A commit linter like commitlint to enforce the convention

These two items would serve as prerequisites for implementing an automated release management solution like:

release-please or

semantic-release

Both options can automatically update the CHANGELOG, create releases, and publish to third-party repositories based on conventional commits.

cc: @anilnatha @rtapella @yunks128 for awareness and input

anilnatha · 2025-05-14T16:55:54Z

@yunks128 The standard to strive for should be the use semantic-release, and perhaps that's the primary recommendation that is promoted by SLIM; however, in practice, there may be challenges with its adoption. Perhaps offering an alternate path an organization can follow that leads to the use of semantic-release would be helpful. Correct me if I'm wrong @riverma , but the recommendation about versioning and changelogs in PR #188 was to began that journey for organizations that aren't ready to fully adopt semantic-release. That was my take away from the meetings we had with Brian and Rob about adopting semantic-release for MDPS, notably, we do want to do it, but not right now due to the impact it would have on all the teams.

stirlingalgermissen · 2025-05-14T16:58:28Z

Pypi/pip supports prereleases. Could also just setup a custom repo for nightly/test releases
The release process should be automated/captured as code and then reviewed by the team like other PRs
+1 for semantic-release/release-please and https://semver.org/. I don't believe we want to invent a new versioning scheme

anilnatha · 2025-05-14T17:24:49Z

Just so we're all on the same page, semver is what @riverma and I recommended in #188.

jpl-jengelke · 2025-05-14T19:21:05Z

Semver and semantic-release shouldn't be mutually exclusive. For Python, for instance, prereleases are standardized with a certain format (from PEP440).

riverma · 2025-05-21T00:15:43Z

Agree with the above!

I see a set of steps involved here:

Write up a step-by-step process for releasing that developers understand, including automation tools release-please and semantic-release
Encourage devs to use https://www.conventionalcommits.org/en/v1.0.0/ to ensure releaseplease or semantic-release can be invoked (they depend on certain types of keywords in git commit messages)
Include any SLIM infusable assets that help make the use of the release process above easier

Draft guide with release template and aggregator script automation

d9c3457

riverma self-assigned this Mar 26, 2024

riverma requested review from a team March 26, 2024 03:44

riverma added the information sharing Process improvements involving documentation, training, guides, communication, etc label Mar 26, 2024

Fixed text typos and other updates

04ce4ec

riverma added a commit that referenced this pull request Mar 26, 2024

Create release.yml

c87cab7

Standardized release template for SLIM as described in #149

nutjob4life reviewed Mar 26, 2024

View reviewed changes

docs/guides/documentation/releases/README.md Show resolved Hide resolved

nutjob4life requested changes Mar 26, 2024

View reviewed changes

jpl-jengelke suggested changes Mar 26, 2024

View reviewed changes

pogi7 reviewed Mar 27, 2024

View reviewed changes

riverma added 2 commits March 29, 2024 16:40

Make curl args more readable

e5beb8e

Standard out language fix

243d920

riverma changed the title ~~[DRAFT] Release Best Practices Guide~~ Release Best Practices Guide Apr 4, 2024

riverma mentioned this pull request Jun 25, 2024

[New Website Feature]: Improve best practice guide template based on markdown recommendations #161

Open

riverma mentioned this pull request Aug 30, 2024

New Guidance on Creating Documentation Organization Hierarchy #30

Closed

yunks128 approved these changes Oct 10, 2024

View reviewed changes

riverma mentioned this pull request May 13, 2025

Pull Request Guide: Best Practice Improvements #188

Merged

riverma mentioned this pull request May 21, 2025

[New Feature]: Support for infusing pull requests best practices NASA-AMMOS/slim-cli#34

Open

9 tasks

riverma added high complexity Ticket has multiple difficult sub-tasks most requested Highly requested by community members labels Jun 5, 2025

Release Best Practices Guide #149

Are you sure you want to change the base?

Release Best Practices Guide #149

Uh oh!

Conversation

riverma commented Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Proposed Changes

Issues

Testing

Uh oh!

riverma commented Mar 26, 2024

Uh oh!

Uh oh!

nutjob4life left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

nutjob4life Mar 26, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jpl-jengelke left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

riverma commented Apr 3, 2024

Uh oh!

mike-gangl commented Apr 12, 2024

Uh oh!

stirlingalgermissen commented Jul 11, 2024

Uh oh!

riverma commented Jul 16, 2024 via email

riverma commented Mar 26, 2024 •

edited

Loading

nutjob4life Mar 26, 2024 •

edited

Loading

anilnatha commented May 14, 2025 •

edited

Loading