Data Pipeline integrates really well with EMR, and it's easy to deploy pipelines via cloudformation making automation possible. We use it to manage complex map-reduce workflows and it usually works pretty smoothly.
It really should be redesigned as a json blob is a terrible way to organize SQL jobs. Why can't things be scheduled like a calendar or meeting request?
The best features according to me are Hybrid Data Integration, Data Movement, Orchestration and Scheduling and Integration with other Azure Services.
SOmetimes it becomes difficult to comprehend the errors due to which the data pipeline fails. Even after looking on internet doesn't help so may be the error message can be improved which helps users to comprehend and easily resolve it.
Data Pipeline integrates really well with EMR, and it's easy to deploy pipelines via cloudformation making automation possible. We use it to manage complex map-reduce workflows and it usually works pretty smoothly.
The best features according to me are Hybrid Data Integration, Data Movement, Orchestration and Scheduling and Integration with other Azure Services.
It really should be redesigned as a json blob is a terrible way to organize SQL jobs. Why can't things be scheduled like a calendar or meeting request?
SOmetimes it becomes difficult to comprehend the errors due to which the data pipeline fails. Even after looking on internet doesn't help so may be the error message can be improved which helps users to comprehend and easily resolve it.