Как собрать платформу обработки данных «своими руками»? / Хабр
https://habr.com/ru/company/itsumma/blog/679516/

Ссылки на YouTube и презентации с Airflow Summit 2022

https://airflowsummit.org/sessions/2022/

GitHub - mindsdb/mindsdb: A low-code Machine Learning platform to help developers build #AI solutions
https://github.com/mindsdb/mindsdb

Доступ к ChatGPT
Если не знаете, как получить доступ к ChatGPT, то советую заюзать https://onlinesim.io/v2/numbers/

Не реклама (сам несколько раз использовал)

Zero-ETL, ChatGPT, And The Future of Data Engineering | by Barr Moses | Apr, 2023 | Towards Data Science
https://towardsdatascience.com/zero-etl-chatgpt-and-the-future-of-data-engineering-71849642ad9c

GitHub - AppFlowy-IO/AppFlowy

AppFlowy is an open-source alternative to Notion. You are in charge of your data and customizations. Built with Flutter and Rust.

33.4k stars ⭐

https://github.com/AppFlowy-IO/AppFlowy

Hi everyone!

Great work on the projects! Now it's time to evaluate your peers.

We've updated the page with the projects (https://github.com/DataTalksClub/data-engineering-zoomcamp/blob/main/cohorts/2023/project.md), now it contains two more links:

- Peer review assignments: https://docs.google.com/spreadsheets/d/e/2PACX-1vRYQ0A9C7AkRK-YPSFhqaRMmuPR97QPfl2PjI8n11l5jntc6YMHIJXVVS0GQNqAYIGwzyevyManDB08/pubhtml?gid=0&single=true
- Evaluation form: https://forms.gle/1bxmgR8yPwV359zb7

To find the projects assigned to you, use the first link (peer review assignments) and find your hash in the first column. You will see three rows: you need to evaluate each of these projects. For each project, you need to submit the form once, so in total, you will make three submissions.

Use this as an opportunity to learn from your peers - and you will learn a lot.

But also remember - if you don't do peer review, you will fail your projects.

Have fun!

Also - the form for submitting project attempt #2 is open, so if you didn't have time to work on your project yet, now you can do it.

https://docs.google.com/spreadsheets/d/e/2PACX-1vRYQ0A9C7AkRK-YPSFhqaRMmuPR97QPfl2PjI8n11l5jntc6YMHIJXVVS0GQNqAYIGwzyevyManDB08/pubhtml?gid=0&single=true Есть где поковыряться)

GitHub - tabixio/tabix: Tabix.io UI

Open source simple business intelligence application and sql editor tool for Clickhouse.

https://github.com/tabixio/tabix

https://www.phind.com/ - Phind: The AI search engine for developers.

Get instant answers, explanations, and examples for all of your technical questions.

Youtube PlayList: Apache NiFi с нуля за 3 часа. Конструктор вместо кода

1. Apache NiFi Install v1.14.0
2. Apache NiFi. Введение и первый опыт на практике. Это же конструктор!
3. Процессоры в Apache NiFi и не только. Разбираемся с деталями конструктора
4. Apache NiFi особенности. Без чего не выйти в прод
5. Проблемы с Apache NiFi и как с ними бороться. Наш опыт
6. Как ускорить процессоры в Apache NiFi. Оптимизация
7. История изменений данных в Apache NiFi. Data Provenance
8. Apache NiFi Registry Install v1.14.0
9. Экспорт и импорт потоков с помощью рук, Rest Api и Apache NiFi Registry
10. Git, Apache NiFi Registry и CI/CD
11. Обзор Keycloak и настройка SSO в NiFi
12. Выкачиваем код Apache NiFi. Меняем под себя и делаем UI для процессора

https://www.youtube.com/playlist?list=PL4MpKy3QjNp_rOEEibc4Ro8UK4g8vLX6_

AvitoTech Team PlayBook

Открытый справочник по ценностям, бизнес-процессам, стандартам, процедурам и правилам, которые используются в команде разработки в Авито.

https://github.com/avito-tech/playbook

Подсмотрел в канале @rtdlinks

Spark Connect Available in Apache Spark 3.4 - The Databricks Blog
https://www.databricks.com/blog/2023/04/18/spark-connect-available-apache-spark.html

Best practices for caching in Spark SQL | by David Vrba | Towards Data Science
https://towardsdatascience.com/best-practices-for-caching-in-spark-sql-b22fb0f02d34

Clickhouse datasource for grafana

Altinity ClickHouse datasource plugin provides a support for ClickHouse as a backend database.

Initially plugin developed by Vertamedia, maintaned by Altinity since 2020.

https://github.com/Altinity/clickhouse-grafana