Data Governance solution for Databricks, Synapse and ADLS gen2

Question

I\'m new to data governance, forgive me if question lack some information.

Objective

We\'re building data lake & enterprise data warehouse from scratch for

Answer 1

Azure Purview is a new service and it would fit your data governance needs well. It is currently (2020-12-04) in public preview. It contains features you are looking in your question, e.g data lineage, and works well with the Azure services you are using (Synapse, Databricks, ADLSg2).

Purview is not a cloud agnostic solution. It exposes Apache Atlas API so some core capabilies and integrations could be run in any cloud. I would still categorize Purview as Azure specific solution.

Purview can manage hybrid data, e.g. data on-premise or other clouds. This way it is agnostic on where your data is. If you need to have some data or use-cases outside Azure, Purview will be able to manage these data assets too.

I saw that data quality features are on the Purview roadmap and will be available later. Also other governance topics will be covered later, e.g. policies.

More info on Purview here: https://azure.microsoft.com/en-us/services/purview/