ADR Suggestion Dependent Parameter serialization

[rozyczko]

ADR proposal: Dependency Serialization for Parameters

Context

In the corelib library, Parameter objects can have dependencies on other parameters through dependency expressions and maps. When serializing these parameters to dictionaries or JSON for persistence, it's crucial to preserve these dependency relationships so they can be accurately reconstructed during deserialization.

Requirements

1. Dependency Preservation: Serialize parameter dependencies (expressions and maps) along with parameter data
2. Reliable Deserialization: Reconstruct dependency relationships after deserialization
3. Order-Independence: Allow parameters to be loaded in any order without breaking dependencies
4. JSON Compatibility: Support full JSON serialization/deserialization
5. Performance: Minimize serialization overhead and deserialization time
6. Stability: Ensure dependency resolution works consistently across serialization cycles

Decision

Implement dependency serialization using a transient serializer_id attribute on Parameter and DescriptorNumber objects. This attribute serves as a stable identifier during serialization/deserialization cycles and is automatically generated and assigned when a Parameter/DescriptorNumber gets used in a dependency.

Implementation Details

• Add a __serializer_id attribute to variables when they get used in a dependency.
• Store dependency maps using __serializer_id values instead of object references or unique_names in the serialized object.
• Use __serializer_id as lookup key during deserialization to recreate dependency.
• Clean up transient attributes, including __serializer_id, after dependency resolution
  , using the _attach_observer method of the observer_pattern described in Dependent Parameters #121.

Alternatives Considered

1. Rely Solely on unique_name

Description: Continue using the existing unique_name attribute for dependency resolution.

Pros:

• Simple implementation
• No changes to core parameter structure
• Backward compatible

Cons:

• Unreliable when unique_name conflicts or is modified when using auto-generated unique_names.
• No guarantee of uniqueness across different serialization contexts.

Why Not Chosen: Fails requirement for reliable deserialization.

2. Store Full Parameter Objects in Dependencies

Description: Serialize complete parameter objects within dependency maps instead of references.

Pros:

• Complete dependency information preserved
• No lookup required during deserialization

Cons:

• Circular reference issues
• Massive serialization size (parameters duplicated)
• Not JSON-compatible (object references can't be serialized)
• Memory inefficient
• Complex deserialization logic

Why Not Chosen: Violates JSON compatibility and performance requirements.

3. Use Persistent UUIDs

Description: Assign permanent UUIDs to parameters for identification.

Pros:

• Globally unique identifiers
• Stable across sessions and environments

Cons:

• Persistent storage requirement (UUIDs saved with parameter data)
• Increased storage size
• Not transient (permanent addition to parameter state)

Why Not Chosen: Fails transient requirement and adds unnecessary persistence overhead.

4. Combining unique_name with UUIDs

Description: Append shortened UUIDs to unique_names instead of increasing integer when assigning default unique_names.

Pros:

• Globally unique identifiers
• Stable across sessions and environments
• Simple implementation
• No changes to core parameter structure

Cons:

• unique_names becomes harder to read, against ADR Suggestion, replace UUIDS with string names in BORG #16
• unique_names cannot be set by the user if used as serialization id
• Risks clashing when deserializing if UUID-part too short.

Why Not Chosen: We want unique_names to be easily readable and to be possible to be set by the user: #16.

Rationale for Chosen Solution

The transient __serializer_id approach provides the best balance of all requirements:

1. Order-Independence: By using stable IDs generated only for serialization, parameters can be loaded in any order
2. Reliability: IDs are guaranteed to be unique within a serialization cycle and stable during the process
3. JSON Compatibility: IDs are simple strings that serialize perfectly to JSON
4. Performance: Minimal overhead (single attribute per parameter)
5. Transience: Attribute is temporary and cleaned up after use, not persisted
6. Simplicity: Leverages existing serialization infrastructure with minimal changes

Consequences

Positive

• Enables reliable serialization of complex parameter hierarchies
• Supports JSON-based persistence and transmission
• Allows for order-independent deserialization
• Minimal performance impact

Negative

• Slight increase in serialization complexity
• Additional transient state management during serialization cycles
• Requires careful cleanup of transient attributes

Risks

• Potential for __serializer_id collisions if not properly managed (mitigated by UUID generation)
• Dependency on proper cleanup to avoid memory leaks (handled by resolution utilities)

Implementation Notes

• serializer_id is generated using UUID4 for uniqueness
• Stored in __serializer_id to avoid conflicts with user attributes
• Dependency maps store serializer_id to deserialize.
• Resolution utilities handle bulk dependency reconstruction
• Error handling provides clear messages for resolution failures

[damskii9992]

When going through the various alternatives considered here and arriving at the final solution, it was also decided that unique_names will be session-only id's. I.e. they are not persistent across sessions and will not be saved upon serialization unless explicitly set by the user.
This means that we have a clear separation of responsibilities:

serializer_ids

Are transient and meant ONLY for serialization/de-serialization identifikation.

unique_names

Are persistent but are NOT serialized. Is meant to be used for runtime references or identification.