Skip to content

Project Merge Algorithm

Joe Clark edited this page Sep 29, 2025 · 1 revision
Algorithm to merge one workflow (the source) into another (the target)

Eg, when merging `staging` into `main`, `staging` is the source and `main` is the `target`

The exercise here is basically to map the source nodes to UUIDS in the target workflow

MERGE(sourceWorkflow, targetWorkflow):

Note: in the CLI, merge returns a new workflow and does not mutate source or target.

- Build an index of of all the nodes in the target workflow, by id (in CLI this means name, not UUID)
- Build a POOL of nodes in the workflow. No sorting required.
- Call MATCH_SOLID to map trivial nodes into an ID Map 
- For each node in the POOL:
  - Call MATCH_EXPRESSION to map nodes with the same expression
  - Call MATCH_PARENT to map nodes with the same structural position in the workflow
  - Call MATCH_POSITION to map nodes with the same layout location
-  At the end of the loop, If a) there are still nodes left, AND b) new ids were added to the ID MAP in the last iteration, run the loop again
- After the pool is exhausted, or all ids have been mapped, any nodes still in the pool need a new UUID
- Call MAP_EDGES with the latest version of the target workflow

MATCH_SOLID(sourceNodes, targetNodes)
  - For each node in the source pool
  - Look for a single node in the target pool with the same name (in CLI, use `id` instead of `name`)
  - If there's a match, call REPLACE_NODE
  - If there are two source nodes with the same id... error? Or continue?

MATCH_EXPRESSION(sourceNodes, targetNodes)
  - For each Step type node in the source pool
  - Look for a single node in the target pool with the same expression
  - (later) Attempt a fuzzy match, rather than an exact one
  - If there's a single match, call REPLACE_NODE
  - If there are multiple matches, call MATCH_PARENT and MATCH_ADAPTOR to find a match

MATCH_PARENT(sourceNodes, targetNodes)
  - For each Step type node in the source pool
  - Look for a single node in the target pool with the same parent
  - If there are multiple matches, call MATCH_ADAPTOR to find a match

MATCH_ADAPTOR(sourceNodes, targetNodes)
  - For each Step type node in the source pool
  - Look for a single node in the target pool with the same adaptor
  - If there's a single match, call REPLACE_NODE

MAP_EDGES(sourceWorkflow, targetWorkflow):
  - for each edge in the source
  - identify the new from/to UUIDS for the edge
  - In the target, find an edge with the same from/to UUIDS
  - Replace the target edge properties with the source edge propertes (excluding any properties?)


MATCH_EDGES()
  - for each edge in the target
  - Get the UUIDs of the source and target
  - Find the corrsponsing
  - Call REPLACE_NODE

REPLACE_NODE(source, target):

  Replace the values of the target node with the values of the source (or, apply the properties of the source to the target)

  - clone the source node into a new node
  - apply the UUID from the target node into the new node
  - (copy other properties from the target, maybe some options? Maybe the credential?)
  - Remove the source and target node form both pools

Clone this wiki locally