Field XML Auditing #8
Conversation
- use rst for docstrings
…gram - Add audit level translation and validation - Support field and process auditing - Generate process diagram for audited fields - Improve error handling and logging
- Fix audit level string comparison - Add tests for audit functionality - Update ruff dependency in pyproject.toml - Improve audit field function - Add test for audit save results
- Skip when audit_data empty - Add tests for audit levels and failure cases - Save if `Field.run` fails
Options shown in system config didn't match the actual strings used in code. And added a warning that currently there's an issue with the "Processes" auditing level
|
Tests are passing through GitHub Actions, but are a few are failing for me related to graphing. Might be related to the issues I'm having running the "Processes" auditing locally as well. |
lloyd-rmi
left a comment
lloyd-rmi
left a comment
There was a problem hiding this comment.
I'm running into some issues locally, but I am able to get the basic Field auditing functionality to work, which is the main thing I want to focus on. Would need to understand why Processes is running into issues for me and confirm if it's an actual bug or just something that I'm missing locally. But that can be worked on after this change.
|
|
||
| if attr_name in original_attrs: | ||
| source = "input" | ||
| elif attr_name in SmartDefault.registry: |
There was a problem hiding this comment.
Just making sure I understand the logic: all provided attributes are captured in the original_attrs variable so then if an attribute shows up outside of that, we can check against the registry if it was possible to supply it with a smart default, correct? So the order of operations matters here.
Note
This feature was created from the
rmi-pip-installbranch and should be reviewed/merged after #7 has been completed. Similarly, this can be edited/merged as needed in @mbarlow12 's absence.Description
This PR introduces an attribute-auditing framework for OPGEE models.
Auditing can now be enabled at four granularities (
none,field,processes,all) and:input,static_default,smart_default,unknown) of every field‐level attribute.field_audit.csv) alongside existing result artifacts.Motivation and Context
Model transparency is critical when reconciling results with known external data and for building an understanding of the underlying model. By recording where every final attribute value originates—and, when requested, how it flows through the process graph—users can:
Types of changes (check all that apply)
Change Summary
Core
opgee/audit.py(new):AuditFlag,AuditRow, level translation/validation helpers._generate_field_audit_report()inspectsAttrDefs, original XML, smart-default registry, etc.audit_field()orchestrates reporting and diagram generation.opgee/field.py– addaudit_dataslot toFieldResult.opgee/manager.py_run_fieldandsave_results().audit_dfs, writefield_audit.csvonly when data present.dask.distributed.TimeoutError.opgee/built_ins/run_plugin.py– propagate-o/--output-dirto global config so diagrams save in the right place.opgee/etc/system.cfg– newOPGEE.AuditLevelsetting with inline documentation.Tests
tests/test_audit.py(>150 LOC) covers:Field.runraises.tests/files/audit_model.xmland minor additions totests/files/test_model.xml.Misc
ruff>=0.11.8in dev extras.Additional notes
None, so existing workflows are unaffected.write_process_diagram()—no extra dependencies introduced.OPGEE.AuditLevelinopgee.cfgor with--set OPGEE.AuditLevel=Field(etc.).Commit list
Field.runfails