We also recently had a workshop with dlt on AI-assisted data ingestion.

Watch the recording and check out the code if you missed it.

Practice what you learned in the homework assignment, and submit it here.

Today at 16:00 CET (in 90 minutes) we have office hours

You can use this link if you want to ask any question in advance:

https://app.sli.do/event/r22cR71X67cQ7gHacVu7LG

See you soon!

We're starting in 4 minutes:

Stream: https://www.youtube.com/watch?v=o28p3Y2CS9A
Questions: https://app.sli.do/event/r22cR71X67cQ7gHacVu7LG

Watch now or later in recording

This is the link I promised to share:

https://getbruin.com/zoomcamp-project/

You get a chance to win Claude Code subscription if you use Bruin in your project

The course management platform is currently down

I'm working on restoring it

Update: I've spent one hour on a call with AWS and they said they needed more time to restore the database.

Hopefully tomorrow it'll get sorted out

The course management platform is back!

If you submitted any homework yesterday, the data may be lost. Please check your homework submissions.

Apologies for the inconvenience!

I'm working on the incident report and will publish it soon. TLDR: don't let your AI agents do terraform apply -auto-approve

This week, we're starting Module 6: Batch Processing.

Reminder: the previous homework deadline is in less than 24 hours.

In this module, you'll learn how batch processing works with Spark and PySpark.

You'll cover:

• Batch processing fundamentals and Spark basics
• Installing and running Spark locally or in Colab
• Working with Spark SQL and DataFrames
• Handling schemas and processing NYC taxi data
• How Spark clusters, joins, and groupBy work internally
• Running Spark in the cloud with Dataproc and BigQuery

Homework deadline: 10 March, 12 AM CET

We also recently had a workshop with dlt on AI-assisted data ingestion. Watch the recording and check out the code if you missed it. Practice what you learned in the homework assignment, and submit it here.

We uploaded the video about creating projects to YouTube, as some of you reported having problems with watching in on Loom

Here it is: https://www.youtube.com/watch?v=BL0E8xO8OnE

We'll also update it in the repo

And by the way, don't forget about our docs: https://datatalks.club/docs/courses/data-engineering-zoomcamp/

There's a lot of useful information there

We're starting our stream about streaming!

This is going to be a part of module 7 about streaming - and reworked workshop from the last year with the latest versions of PyFlink

Stream: https://www.youtube.com/watch?v=YDUgFeHQzJU
Workshop: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/07-streaming/workshop

Watch now or later in recording!

https://alexeyondata.substack.com/p/how-i-dropped-our-production-database

The post about the database incident last week

We're starting module 7 on stream processing.

Reminder: the previous homework deadline is in less than 24 hours.

The materials were created by Zach Wilson, who ran a Flink stream for the course last year. Alexey recorded an updated Apache Flink workshop to reflect support for Flink 2.x and modern Python versions (3.12, 3.9, 3.8).

It covers:

• Streaming fundamentals in Data Engineering Zoomcamp
• Kafka/Redpanda, Python producers and consumers
• Writing streaming events to PostgreSQL
• Apache Flink setup with Docker
• Flink jobs for stream processing
• Windowing, watermarks, and late events
• Real-time aggregations with Flink

Homework deadline: 17 March, 12 AM CET

Learn here: https://github.com/DataTalksClub/data-engineering-zoomcamp/tree/main/07-streaming

I just realized we haven't shared the homework for streaming

https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2026/07-streaming/homework.md

The deadline is moved to 20 March

It’s time to apply everything from the course.

Build your final project with a complete end-to-end data pipeline.

It takes you through the full workflow:

🔸 Choose a dataset you’re interested in
🔸 Build a pipeline to ingest data into a data lake
🔸 Move the data to a data warehouse
🔸 Transform the data to prepare it for analysis
🔸 Build a dashboard to visualize the results

🎥 Watch the Projects how-to video for the full walkthrough and start building.

🏆 Tip: Use Bruin in your project to participate in the competition and win prizes. Details below.

Build your project using Bruin for ingestion, transformation, orchestration, and analysis, share it with the community, and compete for prizes.

Prizes:

🔸 Mac Mini for an outstanding project
🔸 1 year Claude Pro for the top 3 projects
🔸 1 month Claude Pro for participants

To participate:

🔸 Build your Zoomcamp project using Bruin
🔸 Publish it on GitHub with a README
🔸 Share it in #projects on Slack

Winners will be determined by community votes on Slack.

Learn more here: https://getbruin.com/zoomcamp-project/

A quick update: the Bruin project competition is now open to everyone!

This means your submission does not have to be the same as your Zoomcamp final project.

Even if you are not participating in the course, or if you are using other tech for your Zoomcamp final project, you can still build a separate project with Bruin, submit it to the competition, and compete for prizes.

Prizes:

🔸 Mac Mini for an outstanding project
🔸 1 year Claude Pro for the top 3 projects
🔸 1 month Claude Pro for participants

Deadline: Monday, June 1st, 12:00 UTC

More details here: getbruin.com/competition

Great job working on your projects!

Now it's time to learn from your peers

If you submitted your project for attempt, you will find your review assignments here:

https://courses.datatalks.club/de-zoomcamp-2026/project/project1/eval

If not - you still have time for attempt 2. Have fun!

We have just scored project attempt 1

Congratulations to the 269 course participants who passed it! We will release the certificates for you later along with the second batch

If you haven't passed it, you can improve it and submit one more time

Note that if you did pass the project, submitting it again (even with improvements) is considered self-plagiarism. Please don't do it. But you can submit another project if you want

Also if you haven't made a submission for attempt 1, attempt 2 is still open:

https://courses.datatalks.club/de-zoomcamp-2026/project/project2

Have fun building!

The form for submitting the second attempt is now closed.

Now time to learn from your peers!

If you submitted a project for attempt 2, you'll find the project review assignments here:

https://courses.datatalks.club/de-zoomcamp-2026/project/project2/eval

Have fun!