- Introduction
-
About this Course
- Foundry Data Pipelines
-
Intro to Data Pipelines
-
Sample Data Pipeline
-
Exercise Summary
- Getting Started with Data Pipelines
-
When to think, "Pipeline"!
-
Pipeline Stages
-
Creating your Project Structure
-
Exercise Summary
- Foundry Data Pipeline Applications
-
Prototyping
-
Transforming
-
Maintaining
-
Transforming data on the DATAENG learning path
-
Exercise Summary
- Conclusion
-
Key Takeaways
-
Next Steps
DATAENG 01: Data Pipeline Foundations
Get started with Foundry vocabulary and applications used to transform your data.
Authoring data pipelines is traditionally a slow, messy and expensive undertaking that often results in a patchwork of unmanageable pipelines that are:
- expensive to maintain
- inaccessible to non-technical users
- maintained via tribal knowledge.
This course will introduce you to Foundry vocabulary and applications you’ll be using as you transform data.
⚠️ Course Prerequisites
- Foundry Foundations
- Intro to Foundry
⚠️ Note that this course will be split into two tracks:
1. Pipeline Builder (course prefix "Builder")
2. Code Repositories (course prefix "Repositories")
You are of course free to follow both tracks, but we advise you to prioritize the "Builder" track, first.
! Courses without prefix are shared among both tracks.
Learning Objectives
- Understand the definition and essential anatomy of a Foundry data pipeline.
- Learn the sequence of data integration applications you’ll be using through the learning path.
Foundry Skills
- Explore a pipeline with the Data Lineage application.
- Create a project structure for the first stage of your data pipeline.