Airbyte provides a straightforward way to ingest data from different sources into our data warehouse. The biggest benefit is it allows us to skip writing custom ETL code for each data source.
1. Usage is not very transparent it goes up and down without explaination 2. The schema created in the destination is not very easy to read.
It's open-source and community-supported, you can build anything you want, from simple file ingestion to Kafka, S3, etc... The ability to create Process groups and isolate your workloads. The number of prebuilt processors. The flow-based programming comes...
Tracking lineage at a row level is important in data lake ingestion implementation. Can Lineage be controlled at per-row level? Batch transformation performance. Need to Benchmark. May require Kafka
Airbyte provides a straightforward way to ingest data from different sources into our data warehouse. The biggest benefit is it allows us to skip writing custom ETL code for each data source.
It's open-source and community-supported, you can build anything you want, from simple file ingestion to Kafka, S3, etc... The ability to create Process groups and isolate your workloads. The number of prebuilt processors. The flow-based programming comes...
1. Usage is not very transparent it goes up and down without explaination 2. The schema created in the destination is not very easy to read.
Tracking lineage at a row level is important in data lake ingestion implementation. Can Lineage be controlled at per-row level? Batch transformation performance. Need to Benchmark. May require Kafka