ASCON Alpaca always safe despite of unsafe conditions

[si6pack]

First of all, I want to thank you for this great piece of work! It took some time, but now it is up and running on a TJ Allsky camera driven by a raspi.
I did the training for my setting as recommended, however, I come across an issue with respect to the ascom alpaca driver: It reports "SAFE" while the condition is "overcast" or "mostly cloudy" and shoould be "UNSAFE"

The reported confidence level was >80% (I was looking at the log via the "docker logs -f simple-cloud-detect" command - see below.

Unsafe switch conditions were at 10% for "overcast" and "mostly cloudy" in the safety classification tab. I would expect a switch to "UNSAFE", but it got stuck with "SAFE" in the ASCOM status.

Another observation is, that the Safety History rested at two entries...

Here a log entry:
2026-01-13 22:39:20,139 - INFO - Detection: {'class_name': 'mostly cloudy', 'confidence_score': 83.98, 'Detection Time (Seconds)': 0.76, 'image': <PIL.Image.Image image mode=RGB size=3552x3552 at 0x7F7DF0E190>}
2026-01-13 22:39:20,277 - INFO - Published HA states: mostly cloudy, 83.98%, 0.76s
2026-01-13 22:39:20,277 - INFO - Cloud detection: mostly cloudy (84.0%)
2026-01-13 22:39:51,377 - INFO - Detection: {'class_name': 'mostly cloudy', 'confidence_score': 93.7, 'Detection Time (Seconds)': 0.78, 'image': <PIL.Image.Image image mode=RGB size=3552x3552 at 0x7F7C208090>}
2026-01-13 22:39:51,528 - INFO - Published HA states: mostly cloudy, 93.7%, 0.78s
2026-01-13 22:39:51,529 - INFO - Cloud detection: mostly cloudy (93.7%)
2026-01-13 22:40:22,660 - INFO - Detection: {'class_name': 'mostly cloudy', 'confidence_score': 89.37, 'Detection Time (Seconds)': 0.78, 'image': <PIL.Image.Image image mode=RGB size=3552x3552 at 0x7F7D57A4D0>}
2026-01-13 22:40:22,808 - INFO - Published HA states: mostly cloudy, 89.37%, 0.78s
2026-01-13 22:40:22,809 - INFO - Cloud detection: mostly cloudy (89.4%)
2026-01-13 22:40:54,056 - INFO - Detection: {'class_name': 'mostly cloudy', 'confidence_score': 92.57, 'Detection Time (Seconds)': 0.91, 'image': <PIL.Image.Image image mode=RGB size=3552x3552 at 0x7F7C1C7390>}
2026-01-13 22:40:54,199 - INFO - Published HA states: mostly cloudy, 92.57%, 0.91s
2026-01-13 22:40:54,200 - INFO - Cloud detection: mostly cloudy (92.6%)
vercast" or "mostly cloudy"2026-01-13 22:41:25,191 - INFO - Detection: {'class_name': 'mostly cloudy', 'confidence_score': 99.47, 'Detection Time (Seconds)': 0.73, 'image': <PIL.Image.Image image mode=RGB size=3552x3552 at 0x7F7C208090>}
2026-01-13 22:41:25,319 - INFO - Published HA states: mostly cloudy, 99.47%, 0.73s
2026-01-13 22:41:25,320 - INFO - Cloud detection: mostly cloudy (99.5%)
2026-01-13 22:41:56,330 - INFO - Detection: {'class_name': 'mostly cloudy', 'confidence_score': 99.88, 'Detection Time (Seconds)': 0.75, 'image': <PIL.Image.Image image mode=RGB size=3552x3552 at 0x7F7C1DDCD0>}
2026-01-13 22:41:56,462 - INFO - Published HA states: mostly cloudy, 99.88%, 0.75s
2026-01-13 22:41:56,462 - INFO - Cloud detection: mostly cloudy (99.9%)
2026-01-13 22:42:27,472 - INFO - Detection: {'class_name': 'mostly cloudy', 'confidence_score': 95.74, 'Detection Time (

Would could be the culprit?

Thank you very much in advance,
Christian

[chvvkumar]

Hi!

From taking a glance at the logs, the issue you are experiencing the driver reporting "SAFE" despite detecting "mostly cloudy" with high confidence is probably a case sensitivity mismatch between the detected class name and the configuration.

The Problem:

Detection:Logs show the detection returning 'class_name': 'mostly cloudy' (lowercase).

Configuration: The internal list of unsafe conditions (and thresholds) expects Title Case strings (e.g., 'Mostly Cloudy'), which is how they are defined in config.py and the UI.

The Check: The code checks if class_name not in self._unsafe_conditions_set. Since 'mostly cloudy' (lowercase) is not in the set {'Mostly Cloudy', 'Overcast', ...}, the check passes as True, and the driver incorrectly concludes the condition is SAFE.

Could you paste the contents of your labels.txt? I will take a look at if we can better handle case mismatch if this is indeed the case.

[si6pack]

You nailed it!
As requested, here the content of the initail labels.txt:

0 clear
1 cloudy
2 mostly cloudy
3 overcast
4 rain
5 snow
6 whisps

I changes the content of the file to the following content:

0 Clear
1 Cloudy
2 Mostly Cloudy
3 Overcast
4 Rain
5 Snow
6 Whisps

After that (and removing and restarting the container), it worked like a breeze!

One additional question: How often does the safety history change? Does it only respond to changes from [1] safe to unsafe and vice versa? Or does it also change from i.e. [2] unsafe (mostly cloudy) to unsafe (overcast)?
If it should behave like in scenario 2, then it got stuck here...

However, I want to stress, this is an amazing piece of work which will facilitate my imaging enormously! Many thanks for that!

[chvvkumar]

Nice! I will add a check to the script to handle different capitalizations better, thank you for identifying the bug!

As for the history, It behaves like Scenario 1. Currently, the code only records a history entry when the safety state toggles between SAFE and UNSAFE. I made it this way because the intention of this web page is for the ASCOM part of the functionality where the only two states NINA cares about are SAFE and UNSAFE. I wanted this history to quickly be able to check at a glance when for example, my telescope was parked and when imaging resumed. For historical data for all conditions, I use Home Assistant for saving these states:

[chvvkumar]

Created fix for class name cases to be handled better.

b1778c7

[si6pack]

Hi,
thank you for your efforts! But somehow I implented a bug by myself: As you can see above, I added 7 characteristic (0 to 6) for my training model. However, simple-cloud-detect (scd) investigates as per its code, only six types of characteristics (copied from the initial installation):
0 Clear
1 Mostly Cloudy
2 Overcast
3 Rain
4 Snow
5 Wisps of clouds

This is also reflected in the "Safety Classification" in the Alpaca Server - only the before mentioned classes are used to determine "safe" or "unsafe".

If I got you right (your initial response), scd is only aware of these 6 classes (caps incl.). Addition of the 7th class (in my case "Cloudy") is unknown to the scd code. Therefore, "Cloudy" leads, as per your explanation from above, to "safe" condition.

Today, I repeated my training with nearly 6000 images and I am watching (besides soccer) the safe/unsafe flag in the Alpaca driver when I remarked the "Cloudy" "safe" case.

Consequently, I will repeat my training with 6 classes tomorrow (skip the Cloudy tag and put these data into Mostly Cloudy)...

[chvvkumar]

If I understand the issue correctly, you in essence added a new class without changing the model right? If so, unfortunately you will have to retrain if you change the classes. (Please let me know if I misunderstood!)

But if you mean the UI and the classification is not seeing the new classes, the container should pick up any changes in the labels.txt file automatically on startup.

As for training, you don't have to use full resolution images for training nor for detection. The model will resize the images down to ~300x300 pixels internally any way. It is important that the training image resolution matches the detection images. I train mine at 800x800 and have a module in AllSky that outputs an 800x800 image that this container then uses for detection. Makes training much quicker and less resource intensive.