Databricks – Take dataset and build parent child relationnships
Project detail
I have a dataset which outlines output and input conditions for jobs. Need to parse this out and develop a Parent/Child relationship for all jobs.
Need to understand full upstream and downstream for any job.
Must be able to work via screen share.
Must have extensive knowledge in pyspark and or Databricks and use Jupyter to create a parent-child relationship with nodes and edges.