Data pipelines powered by Open source and fun
The world of data is ever-growing. There are ample technologies out in the market. Through the session, we will see some beautiful open source tools in action. Well knowing tools is often the first step, but indeed an important one. This session is best suited for beginners, keen to dive into data world.
Typical data engineering actions β
-1) Connecting with data sources β‘οΈ
0) Pre-processing π€«
1) Ingestion π
2) Business logic ππ»ββοΈ
3) Dashboarding π
4) Lift and shifts πͺπ½
5) Quality control and statistical checks βοΈ
* Everything else follows…
Often each step adds a special tooling need. Open source is a great way to get started eliminating steep learning curves.
Speaker
-
Sayantika BanikQuansight
Sayantika is a D&I advocate, Open-source contributor, and Data engineer at Quansight. Messing with data is her core strength and giving back to the community is very important to her. More about SayantikaΒ΄s work, contribution – sayantikabanik.com, GitHub