Introducing G2.ai, the future of software buying.Try now

What is the best way to databricks in ADF

How to make a pipeline in ADF using databricks
1 comment
Looks like you’re not logged in.
Users need to be logged in to answer questions
Log In
Ajay Kumar S.
AS
Analyst at Protiviti
0
Firstly, you need to create a Linked Service for ADF to connect to your Databricks Cluster. You can connect to as many clusters as you want. Then create a pipeline in the ADF, in the Activities list click in the Databricks Option and then drag the attribute of Notebook/Python/Jar as per your need. Under General Give the Name of the Stem and then under Azure Databricks Option, you need to specify the cluster name(where your notebook resides) which are connected via Linked Service in first step. Lastly, under the Settings option, you need to navigate to the Databricks Notebook which you want to run and then select it.
Looks like you’re not logged in.
Users need to be logged in to write comments
Log In
Reply