Skip to content

Graph Model for SuperGlue Lineage Services #11

@akumarb2010

Description

@akumarb2010

Is your feature request related to a problem? Please describe.
Currently, superGlue is using RDBMS for graph modeling and lineage services will construct the Graph from metadata. The challenges with this approach is as below.

  1. Handling full property Graph features will become nightmare and not scalable
  2. To handling column lineage will become more complex with current model
  3. Graph property based index is not straight forward.

To handle all the above issues, at Intuit, we have started working Graph model and integration with Neo4j.

Describe the solution you'd like
Coming up with Property Graph schema, integrating SuperGlue with Neo4j.

Describe alternatives you've considered
We will be working on end-end design document with all the details on various approaches

Additional context
NA

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions