Implement tracer_data in CAM-SIMA; registry entries for prescribed ozone/BAM; NaN check in physics data check

[jimmielin]

Tag name (required for release branches):
Originator(s): @jimmielin

Description (include the issue title, and the keyword ['closes', 'fixes', 'resolves'] followed by the issue number):

• Port tracer_data, a generic reader for chemistry and aerosol data, from CAM to CAM-SIMA. The module is as close to the original tracer_data as possible, only removing CAM-specific features (i.e., chunking and physics buffer), FV dycore polar averaging, structured dycore latitude weighting, and a rmv_file flag that deletes files after being read -- closes CCPPize/move tracer_data to CAM-SIMA #432.
• Moves conservative regridding utility src/utils/horizontal_interpolate.F90 to CAM-SIMA (intended to be included in a sparse checkout from CAM in the future) -- currently only used for aircraft inputs, but could be used for all regridded data read from tracer_data in the future.
• Add string utilities increment_string, last_non_digit, and get_last_significant_char in support of tracer_data.
• Add registry entries for prescribed bulk (BAM) aerosol and prescribed ozone for radiation.
• Add tracer_data chemistry and aerosols (BAM) input regression test (Derecho GNU).
• Add a check for NaN values in physics check data -- closes NaNs in model state are not detected by physics data differences check #440.
  If any NaN values are detected in the model state, the difference count is incremented and the maximum difference will be denoted as NaN.
  The number of NaNs will be reported in the output. e.g.,

 ********** Physics Check Data Results **********

 TIMESTEP:            1

 Variable                                          # Diffs  Max Diff   Max Diff loc (rank, col, lev)
 --------                                          -------  --------   -----------------------------
 ocar1                                                  48       NaN   (0,1,25)
 (!) 48 NaN values in variable!
 bcar1                                                  48       NaN   (0,1,25)
 (!) 48 NaN values in variable!
 sslt1                                                  48       NaN   (0,1,25)

Describe any changes made to build system: N/A

Describe any changes made to the namelist: N/A

List any changes to the defaults for the input datasets (e.g. boundary datasets): N/A

List all files eliminated and why: N/A

List all files added and what they do:

A       src/utils/tracer_data.F90
  - new tracer_data utility in CAM-SIMA ported from CAM (will not be moved from CAM)

A       src/utils/horizontal_interpolate.F90
  - move horizontal_interpolate conservative regridder for chemistry data from CAM.

A       cime_config/testdefs/testmods_dirs/cam/outfrq_trcdata_bam_derecho/shell_commands
A       cime_config/testdefs/testmods_dirs/cam/outfrq_trcdata_bam_derecho/user_nl_cam
  - new tracer_data chemistry and aerosols input data regression test.

List all existing files that have been modified, and describe the changes:
(Helpful git command: git diff --name-status development...<your_branch_name>)

M       cime_config/testdefs/testlist_cam.xml
  - new tracer_data chemistry and aerosols input data regression test.

M       src/data/registry.xml
  - add prescribed ozone for radiation.
  - add prescribed BAM aerosol fields for radiation (used for input check).

M       src/physics/utils/physics_data.F90
  - check for NaNs in physics data check.

M       src/physics/utils/physics_grid.F90
  - make public the `dycore_unstructured` logical for tracer_data.

M       src/utils/string_utils.F90
  - new string utilities `increment_string`, `last_non_digit`, and `get_last_significant_char` in support of tracer_data.

If there are new failures (compared to the test/existing-test-failures.txt file),
have them OK'd by the gatekeeper, note them here, and add them to the file.
If there are baseline differences, include the test and the reason for the
diff. What is the nature of the change? Roundoff?

derecho/intel/aux_sima:

derecho/gnu/aux_sima:

If this changes climate describe any run(s) done to evaluate the new
climate in enough detail that it(they) could be reproduced:

CAM-SIMA date used for the baseline comparison tests if different than latest:

[nusbaume]

Thanks for bringing in this major new capability @jimmielin! I have some change requests, some of which would require non-trivial amounts of effort.

However, given that it is unclear what the future of all of this code will be, I am happy if you want to push back on some of my requests, or make them issues that we can tackle in the future. Just let me know!

[nusbaume]

Can we get rid of this subroutine entirely, and instead just use the one from ccpp_tuvx_utils?

[jimmielin]

It's the same subroutine, but ccpp_tuvx_utils is in physics/ncar_ccpp/to_be_ccppized/. Could I confirm if we are OK with CAM-SIMA code depending on atmos_phys code? Thanks!

[nusbaume]

Yep, using atmospheric_physics code in CAM-SIMA is totally allowed. In fact we are doing that already, for example here:

https://github.com/ESCOMP/CAM-SIMA/blob/development/src/dynamics/mpas/dyn_coupling_impl.F90#L509

[jimmielin]

Thanks, updated to use rebin from atmos_phys!

[nusbaume]

Everything looks good to me now, thanks @jimmielin! I think the only outstanding request is to remove the copy of rebin, but I don't think that requires an extra re-review from me. Thanks again!