-
Notifications
You must be signed in to change notification settings - Fork 14
Project Merge Algorithm
Joe Clark edited this page Sep 29, 2025
·
1 revision
Algorithm to merge one workflow (the source) into another (the target)
Eg, when merging `staging` into `main`, `staging` is the source and `main` is the `target`
The exercise here is basically to map the source nodes to UUIDS in the target workflow
MERGE(sourceWorkflow, targetWorkflow):
Note: in the CLI, merge returns a new workflow and does not mutate source or target.
- Build an index of of all the nodes in the target workflow, by id (in CLI this means name, not UUID)
- Build a POOL of nodes in the workflow. No sorting required.
- Call MATCH_SOLID to map trivial nodes into an ID Map
- For each node in the POOL:
- Call MATCH_EXPRESSION to map nodes with the same expression
- Call MATCH_PARENT to map nodes with the same structural position in the workflow
- Call MATCH_POSITION to map nodes with the same layout location
- At the end of the loop, If a) there are still nodes left, AND b) new ids were added to the ID MAP in the last iteration, run the loop again
- After the pool is exhausted, or all ids have been mapped, any nodes still in the pool need a new UUID
- Call MAP_EDGES with the latest version of the target workflow
MATCH_SOLID(sourceNodes, targetNodes)
- For each node in the source pool
- Look for a single node in the target pool with the same name (in CLI, use `id` instead of `name`)
- If there's a match, call REPLACE_NODE
- If there are two source nodes with the same id... error? Or continue?
MATCH_EXPRESSION(sourceNodes, targetNodes)
- For each Step type node in the source pool
- Look for a single node in the target pool with the same expression
- (later) Attempt a fuzzy match, rather than an exact one
- If there's a single match, call REPLACE_NODE
- If there are multiple matches, call MATCH_PARENT and MATCH_ADAPTOR to find a match
MATCH_PARENT(sourceNodes, targetNodes)
- For each Step type node in the source pool
- Look for a single node in the target pool with the same parent
- If there are multiple matches, call MATCH_ADAPTOR to find a match
MATCH_ADAPTOR(sourceNodes, targetNodes)
- For each Step type node in the source pool
- Look for a single node in the target pool with the same adaptor
- If there's a single match, call REPLACE_NODE
MAP_EDGES(sourceWorkflow, targetWorkflow):
- for each edge in the source
- identify the new from/to UUIDS for the edge
- In the target, find an edge with the same from/to UUIDS
- Replace the target edge properties with the source edge propertes (excluding any properties?)
MATCH_EDGES()
- for each edge in the target
- Get the UUIDs of the source and target
- Find the corrsponsing
- Call REPLACE_NODE
REPLACE_NODE(source, target):
Replace the values of the target node with the values of the source (or, apply the properties of the source to the target)
- clone the source node into a new node
- apply the UUID from the target node into the new node
- (copy other properties from the target, maybe some options? Maybe the credential?)
- Remove the source and target node form both pools