Skip to content

Conversation

@mbarlow12
Copy link

@mbarlow12 mbarlow12 commented May 1, 2025

Note

This branch was originally built to integrate with wrapgee to allow RMI folks to easily install OPGEEv4 via pip install -e packages/opgee. Other fixes came about as various bugs were encountered.

@mbarlow12 is OOO next week, please feel free to make push updates to the rmi-pip-install branch and approve/merge when ready.

Description

  • Packaging update – introduce pyproject.toml with full PEP 621 metadata and move to the modern PEP 517 build backend; legacy setup.py, setup.cfg, and requirements.in are removed. Requirements are now generated with uv pip compile, producing a single, lock-style requirements.txt.
  • Misc. repo cleanup – stale artifacts are gone and unused build-release GitHub Action is deleted.
  • Dependency bumps – dozens of libraries (Dash 3, Dask 2025.4, SciPy 1.15, etc.) were updated to latest compatible versions.
  • Minor functional fixes & TODO scaffolding
    • DownholePump: correct ordering of multiply_flow_rates vs. fugitive-addition.
    • TransmissionCompressor / HeavyOilUpgrading: rename stream contents from plain “gas” to explicit “exported gas” / “gas process”.
    • Added clarifying docstring tweak in Stream.flow_rate and several TODO comments around boundary handling, acid-gas splitting, emissions categorisation.
  • Tests & fixtures
    • Global PostProcessor.decache() fixture prevents state bleed between tests.
    • Updated a numeric expectation in test_processes.py; value shifted from 1520.21852 → 1494.86053 mmbtu/day after dependency upgrades.

Motivation and Context

  • Allow contributors and users to simply run pip install -r requirements.txt -e . without relying on Conda or legacy setup.py.
  • Centralise dependency management and lay groundwork for automated wheel/SDist publishing and future CI/CD build steps.
  • Keep the codebase current with Python ≥3.11 and the latest library APIs.

Types of changes (check all that apply)

  • Bug fix (non-breaking)
  • Bug fix (breaking)
  • New feature (non-breaking)
  • New feature (breaking)
  • Docs/comments
  • Test
  • Format

Change Summary

  • Build system: migrate to PEP 621/517; drop legacy packaging scripts.
  • Dependencies: regenerate lock with uv; bump many libs to 2025-series versions.
  • Repo structure: adopt src/ layout, prune obsolete PDM & GitHub-Actions files.
  • Code fixes: small behavioural tweaks (DownholePump flow-rate math, explicit gas stream names).
  • Quality-of-life: fresh docstrings, TODO markers for upcoming boundary/acid-gas work.
  • Tests: new auto-decache fixture; expectation update to reflect tiny numeric drift.

Additional notes

  • TODO: Identify which upgraded library (or combination) caused the ~1.7 % shift in gas-flow test values. No algorithmic changes were made, so we expect this to be numerical precision/constant changes upstream. Track and document the root cause in a follow-up PR.
  • Removing/renaming stream contents (“gas” → “exported gas” / “gas process”) may require downstream XML model updates; confirm with model owners before merging.

Commit List

  • 24df7d4 (2025-05-01) typo
  • 5e9d1f4 (2025-05-01) chore: whitespace
  • 30b383c (2025-05-01) chore: revert auto-formatting and type hints, remove field_to_xml
  • 3b2b68c (2025-05-01) whitespace
  • 16b7a27 (2025-05-01) revert blank lines
  • 142d883 (2025-05-01) revert worker detection and decorator changes
  • 5b381d1 (2025-05-01) typo
  • ee582e7 (2025-05-01) chore: revert import reorganization
  • 982982f (2025-05-01) chore: revert autoformatting and type hinting, remove xml audit
  • 6acc98e (2025-05-01) delete build-release draft action
  • d92af8f (2025-05-01) chore(dependencies): update dependencies and lockfile
  • 72b53de (2025-04-23) formatting and remove upper bound on python version
  • 9b669ae (2025-03-25) pin pint-pandas due to .dot bug
  • 066a3bf (2025-03-21) revert necessary stream contents
  • f9c9626 (2025-03-18) revert stream naming, remove outdated pdm scripts
  • bb4e8d8 (2025-03-18) reorder impute operations to reflect how they're applied in run
  • c4db744 (2025-03-18) add OGSI requested TODO comments
  • 24a878f (2025-03-18) add boundary bug comment
  • 1b20ab1 (2025-03-18) fix: single stream edge case, field/processes not picking up smart defaults feat: clean up final xml saving
  • e7cd502 (2025-02-25) Merge branch 'main' into rmi-pip-install
  • 5f5e54b (2025-02-14) fix failing tests for post proc and subcommand
  • a3ce1f0 (2025-02-14) Merge branch 'main' into rmi-pip-install
  • 573d83b (2025-02-14) update uv.lock
  • 592ab59 (2024-11-22) adjust install
  • 0b743b6 (2024-11-22) revert to pip install (non editable)
  • e0e81f8 (2024-11-22) name update
  • 46face4 (2024-11-22) update ci matrix,
  • 3ad2e8b (2025-01-28) A direct copy of Wennan's commit under the gas-transmission branch that adds the ability to adjust the electric grid mix for fields.
  • 18fd2b2 (2025-01-21) Merge branch 'main' into rmi-pip-install
  • 093aa0d (2024-12-18) add dependencies and update build config
  • ddf12e2 (2024-12-17) remove ini opts for pytest
  • cb4fc67 (2024-12-17) move back to flat layout
  • 0662b65 (2024-12-15) update lock
  • f913ce9 (2024-12-13) update lock
  • 33a3425 (2024-12-13) handle edge case for post plugins
  • d7cd38e (2024-12-13) handle xml saving
  • 86f45c7 (2024-12-13) remove xml saving from init
  • 845501e (2024-12-13) add informative comments
  • 7e857b5 (2024-12-13) comment out XmlSavePathname overwrite
  • 52fdf73 (2024-12-13) fetch relevant items from outdated update/python-3.11 branch
  • fdb9f4e (2024-12-13) Merge remote-tracking branch 'upstream/master' into pip-installable
  • 65ec190 (2024-12-04) Merge remote-tracking branch 'upstream/master' into update/python-3.11
  • d668005 (2024-12-02) Merge branch 'main' into update/python-3.11
  • 87c47c0 (2024-12-02) Merge branch 'main' into update/python-3.11
  • 44e3b83 (2024-11-26) skip non-enabled processes in graph creation, misc formatting
  • fdce903 (2024-11-25) update dependencies
  • cbf21a0 (2024-11-25) Merge remote-tracking branch 'upstream/master' into update/python-3.11
  • 7cc6b42 (2024-11-21) initial attempt at additional type hints
  • 051ca33 (2024-11-21) misc formatting: indentation and import ordering, docstring update
  • 0219f84 (2024-11-19) update lock
  • 9543985 (2024-11-14) Merge branch 'main' into update/python-3.11
  • 7dc5c67 (2024-11-06) Merge branch 'main' into update/python-3.11
  • 3123ecb (2024-11-05) update ignored files
  • 28fb0f7 (2024-11-05) abstract annotation for run method
  • 6120fbc (2024-11-05) update project details
  • a9f83ed (2024-11-05) Merge branch 'fix/gh-actions' into update/python-3.11
  • 39ef693 (2024-10-30) test config moved to pyproject.toml,
  • 0f8af91 (2024-10-30) Merge remote-tracking branch 'upstream/fix/gh-actions' into update/python-3.11
  • 48c7836 (2024-10-30) remove setuptools
  • fc7f8a0 (2024-10-30) src layout, project files

@mbarlow12 mbarlow12 marked this pull request as ready for review May 1, 2025 22:56
@mbarlow12 mbarlow12 requested review from lloyd-rmi and rjplevin May 1, 2025 22:56
@mbarlow12 mbarlow12 changed the title Rmi pip install Minimal pip install (local editable), dependency updates, & management May 2, 2025
@mbarlow12 mbarlow12 mentioned this pull request May 5, 2025
4 tasks
@john
Copy link
Member

john commented May 5, 2025

Thank you @mbarlow12 ! @rjplevin , @lloyd-rmi , is this something you could review and, if appropriate, merge?

pyproject.toml Outdated
]
requires-python = ">=3.11"
readme = "README.md"
license = { text = "MIT License" }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mbarlow12 If I understand correctly, this is implying that the entire repo is under the MIT license. If so, that's not the case, unfortunlately. But it looks like the pyproject spec supports 'license-file' as well as 'license'--so we can just point to the hybrid MIT-Stanford license that's in the repo? Also, that way if we're ever able to change the license, using a pointer means this will always refer to whatever is current. If that's the case, can we make that change, to use license-file rather than license?

@lloyd-rmi
Copy link

lloyd-rmi commented May 7, 2025

For some reason test_graph.py fails for me locally, before I have even made any changes to the PR. It seemed to run fine on the automated tests so I won't block on it, but should figure out why there's a difference there.

lloyd-rmi added 2 commits May 7, 2025 10:35
It was mentioned in PR#7 that the renaming of these stream contents could require further XML changes. Since there is uncertainty here and there is currently no harm in keeping the generic stream names, I'm reversing the renaming for now. This should be revisited when we're confident that the name changes won't result in any unknown knock on effects.
@lloyd-rmi
Copy link

Windows test is failing when trying to install conda, but I assume that's not an issue because we're getting rid of our conda usage anyways.


<Stream src="HeavyOilUpgrading" dst="ProductionBoundary">
<Contains>gas</Contains>
<Contains>gas process</Contains>

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there no consequences to changing the wording here? My assumption is that this controls the naming of streams as they are going through the model so updating the naming here is only relevant for identifying a specific stream during debugging/logging, but has no other effect, correct?

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a commit to reverse this for now simply because there's still a bit of uncertainty around any potential knock on effects and this renaming isn't strictly needed within this PR. Can be revisited later when we can confirm this renaming doesn't affect anything unexpectedly

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Hmm... github marked this as "Outdated", but I see comments from 28 minutes ago.)

The stream content tags are used by processes to distinguish among input / output streams, so it's a key runtime feature, not just for debugging/logging. In general, changes to stream contents must be coordinated with the processes at both ends of the stream. The exception is if each end expects only 1 stream and doesn't look for it by name.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Outdated because I added a commit to reverse the name changes here as they weren't critical to this PR. Still something that we can certainly do, but I just wanted to take away any uncertainty while Michael is OOO this week.

@lloyd-rmi lloyd-rmi merged commit 5048615 into main May 7, 2025
2 of 3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants