Exploring Spotify’s Luigi to build ETL pipeline

Problems with data workflow in production

Nowadays, data is the foundation of the modern business world. Data on its own is not very useful and is often stored in some form of application database that isn’t easy to use for analytics. It’s really common in a company to have to move and transform data, using them for different testing, quality assurance and monitoring in production. This data often comes from multiple sources…

--

--

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store