Handle bad log names and reduce logging

[g7gpr]

This is a fix for issue #793.

When a badly formed log name appears in ~/RMS_data/logs/ this causes LogArchiver to fail.

This PR removes any badly formed log names, and slightly reduces logging.

(vRMS) david@lap-deb-01:~/source/RMS$ python -m Utils.LogArchiver -t
Camera settings file: ./camera_settings.json
Loading the default config!
Log type SETI latest info uploaded was 2025-07-30T14:31:09
Log type camControl latest info uploaded was 2025-08-06T15:10:36
Log type log latest info uploaded was 2025-12-23T06:12:48
Log type reprocess latest info uploaded was 2025-12-20T11:02:20
                 : log
                 : camControl
                 : bad.log
                 : reprocess
                 : log
                 : camControl
                 : SETI
                 : bad.log
                 : reprocess
Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "/home/david/source/RMS/Utils/LogArchiver.py", line 300, in <module>
    archive_file_name = makeLogArchives(config, latest_captured_directory_full_path, update_tracker=cml_args.tracker)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/david/source/RMS/Utils/LogArchiver.py", line 219, in makeLogArchives
    date_for_this_log_type = datetime.datetime.fromisoformat(latest_log_uploads_dict[log_file_type])
                                                             ~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^
KeyError: 'bad.log'

However, with this PR

(vRMS) david@lap-deb-01:~/source/RMS$ python -m Utils.LogArchiver -t
Camera settings file: ./camera_settings.json
Loading the default config!
Log type camControl latest info uploaded was 2025-08-06T15:10:36
Log type log latest info uploaded was 2025-12-23T06:12:48
Log type reprocess latest info uploaded was 2025-12-20T11:02:20
Adding camControl_log_AU000A_20250806_151036.log to log upload archive to ensure overlap with last upload
Adding log_AU000A_20251223_061248_001.log to log upload archive to ensure overlap with last upload
Adding reprocess_log_AU000A_20251220_110220_001.log to log upload archive to ensure overlap with last upload
Log directory structure created at /tmp/tmphtalpifh
For log file type camControl the newest file is camControl_log_AU000A_20250806_151036.log, last entry is 2025-08-06 15:10:36
For log file type log the newest file is log_AU000A_20251223_061248_001.log, last entry is 2025-12-24 02:48:25
For log file type reprocess the newest file is reprocess_log_AU000A_20251220_110220_001.log, last entry is 2025-12-20 11:03:00
Logs archived at /home/david/RMS_data/CapturedFiles/AU000A_20250809_102552_318266/AU000A_20250809_102552_318266_logs.tar.bz2

[markmac99]

A couple of comments:
the logger name shold be 'rmslogger' to be consistent and avoid clashes with generic loggers and 3rd party loggers. Also i think extractDateFromLogName can be simplified and made a bit more pythonic.

[markmac99]

Use the RMS logger name here, to avoid clashes with generic loggers. See most other modules for an example.

# Get the logger from the main module
log = getLogger("rmslogger")

[g7gpr]

Thanks for the tip. I've made this error in other places, so I'll find and fix them all.

[markmac99]

extractDateFromLogName is perhaps overcomplex.
i think for RMS logs we can assume the log is of the form
{purpose}_{stationid}_{ymd}_{hms}_{extras}.log
so it should be possible to ignore the 1st element, check the 2nd is a stationid, and treat the next two as a datetime in the format %Y%m%d_%H%M%S, with an exception handler to manage non-dates and malformed dates: something like

log_name_fields = log_name.split("_")
try: 
  if len(log_name_fields) > 3 and config.stationID.upper() == log_name_fields[1].upper():
    dtstr = f'{log_name_fields[2]}_{log_name_fields[3]}'
    return datetime.datetime.strptime(dtstr, '%Y%m%d_%H%M%S').isoformat()
 except Exception:  # handle malformed dates and non-dates
    pass
return datetime.datetime(2000,1,1,0,0,0).isoformat()

[g7gpr]

I started with code like this, but, alas, not as elegant. I did find some logs with an underscore in the {purpose} field, which broke my first attempt. However, losing those logs to gain simplicity is a good trade.

[g7gpr]

How does this look?

    log_name_fields = log_name.upper().split("_")
    stationID_upper = config.stationID.upper()

    if stationID_upper in log_name_fields:
        index_date = log_name_fields.index(stationID_upper) + 1
        index_time = index_date + 1

        try:
            if len(log_name_fields) > index_time:
                dtstr = f'{log_name_fields[index_date][0:8]}_{log_name_fields[index_time][0:6]}'
                return datetime.datetime.strptime(dtstr, '%Y%m%d_%H%M%S').isoformat()

        except Exception:
            pass


    return ISO_DATE_2000

[g7gpr]

Putting back to draft while testing changes

[g7gpr]

On test on au001b