Apache Spark 3.1.1 ✨ is here!

The changes are mainly about:
- Core
- Spark SQL
- Structured Streaming
- PySpark
- MLlib
- SparkR
and more.

For further details: https://spark.apache.org/releases/spark-release-3-1-1.html.

DataStreamingIT in Italian: @kafkastreamingit

#ApacheSpark #SparkStreaming #PySpark #Python #streamprocessing #datastreamingit

21-23 SEP ApacheCon @ Home 2021. CfP are opened: https://blogs.apache.org/conferences/entry/call-for-presentations-for-apachecon.

#ApacheCon #ASF #ApacheKafka #ApacheFlink #ApacheBeam #ApacheSpark #datastreamingit

Litestream (Real-time S3 replication for SQLite) v0.3.3 ✨ is out!

Released: https://litestream.io/blog/litestream-v0.3.3-released.

Why I [Ben Johnson] Built Litestream: https://litestream.io/blog/why-i-built-litestream.

DataStreamingIT in Italian: @kafkastreamingit

#Litestream #databases #S3 #SQLite #datastreamingit

Apache Kafka 2.8 ✨ without ZooKeeper (KIP-500) first look:

- Quorum Controller
- KRaft aka Kafka RAFT Metadata mode,

explained by Ben Stopford, author of 📚 Designing Event-Driven Systems 📖: https://www.confluent.io/blog/kafka-without-zookeeper-a-sneak-peek.

DataStreamingIT in italiano: @kafkastreamingit

#ApacheKafka #Kafka #KRaft #streamprocessing #distributedsystems #datastreamingit

What is a Streaming Database? 🤔

Brief summary from VentureBeat: https://venturebeat.com/2021/04/04/what-is-a-streaming-database.

Data Streaming IT in Italian: @kafkastreamingit

#ApacheKafka #Kafka #ksqlDB #databases #streamprocessing #datastreamingit

21-23 SEP ApacheCon @ Home 2021.

Registration now opened: https://hopin.com/events/apachecon-2021-home.

Data Streaming IT in italiano: @kafkastreamingit

#ApacheCon #ASF #ApacheKafka #ApacheFlink #ApacheBeam #ApacheSpark #opensource #datastreamingit

Beam College 🎓

Apache Beam 🐝 free training organized by the Community: https://beamcollege.dev.

Data Streaming IT in italiano: @kafkastreamingit

#ApacheBeam #BeamCollege #Beam #streaminganalytics #streamprocessing #dataprocessing #datastreamingit

Apache Flink 🐿 Stateful Functions aka StateFun 3.0.0 ✨ is out!

New features:
- Unified Languages SDKs;
- Cross-Language Type System;
- Dynamic Registration of State and Functions
and more.

For further details: https://flink.apache.org/news/2021/04/15/release-statefun-3.0.0.html.

DataStreamingIT in italiano: @kafkastreamingit

#ApacheFlink #Flink #StatefulFunctions #StateFun #streamprocessing #datastreamingit

Apache Kafka 2.8.0 ✨ is finally out!

- KIP-500: Replace ZooKeeper with a self-managed quorum;
- KIP-700: Add Describe Cluster API;
- KIP-684: Support mutual TLS authentication on SASL_SSL listeners;
- KIP-676: Respect logging hierarchy;
- KIP-673: Emit JSONs with new auto-generated schema;
- KIP-612: Limit broker connection creation rate;
- KIP-516: Topic identifiers.

Kafka Connect
- KIP-661: Expose task configurations in Connect REST API.

Kafka Streams
- KIP-696: Update Streams FSM to clarify ERROR state meaning;
- KIP-689: Extend StreamJoined to allow more store configs;
- KIP-680: TopologyTestDriver should not require a properties argument;
- KIP-671: Introduce Kafka-Streams-specific uncaught exception handler;
- KIP-663: API to start and shut down Streams threads;
- KIP-659: Improve TimeWindowedDeserializer and TimeWindowedSerde to handle window size;
- KIP-572: Improve timeouts and retries in Kafka Streams.

For further details, read the Apache blog post: https://blogs.apache.org/kafka/entry/what-s-new-in-apache5.

Data Streaming IT in Italian: @kafkastreamingit

#ApacheKafka #Kafka #KafkaStreams #streamprocessing #datastreamingit

NATS 2.2 ⚡️ JetStream release is out!

JetStream features:
- At-least-once delivery and exactly once within a window;
- Store data and replay by time or sequence;
- Cleanse specific messages (GDPR);
- Horizontal scalability;
- Persist Streams and replay via Consumers
and more.

For further details: https://nats.io/blog/nats-whats-new-22.

DataStreamingIT in italiano: @kafkastreamingit

#NATS #JetStream #CNCF #golang #streamprocessing #datastreamingit

ksqlDB 0.17 🚀 is here!

CI/CD features, new TIMESTAMP data type and much more.

See: https://twitter.com/ksqldb/status/1386729481507770369?s=21.

DataStreamingIT in italiano: @kafkastreamingit

#ApacheKafka #Kafka #ksqlDB #databases #DevOps #streamprocessing #datastreamingit

Apache Druid 0.21.0 🧙‍♂️is here!

New features:
- Leader election based on Kubernetes (without ZooKeeper);
- New grouping aggregator function;
- Old Kafka version support
and much more.

For further details: https://github.com/apache/druid/releases/tag/druid-0.21.0.

DataStreamingIT in italiano: @kafkastreamingit

#ApacheDruid #Druid #realtimeanalytics #databases #datastreamingit

Apache Flink 1.13.0 🐿 is out!

New features:
- Reactive scaling;
- Flame Graphs;
- Machine Learning Library moved to flink-ml;
- Windows via Time-valued functions;
- PyFlink improvements
and much more.

For further details: https://flink.apache.org/news/2021/05/03/release-1.13.0.html.

Data Streaming IT in italiano: @kafkastreamingit

#ApacheFlink #Flink #streamprocessing #SQL #PyFlink #Python #datastreamingit

Kafka Summit Europe 2021 📅 11-12 MAY 📆 is happening now!

✨ kafka-summit.org

#ApacheKafka #Kafka #KafkaSummit #streamprocessing #datastreamingit

Apache Beam 🐝

Beam Summit 2021 📅 04-06 AUG 📆

Free registration and CfP: https://beam-summit-2021-digital.web.app.

Data Streaming IT in italiano: @kafkastreamingit

#ApacheBeam #Beam #BeamSummit #streaminganalytics #dataprocessing #datastreamingit

Just won 📚Mastering Kafka Streams and ksqlDB: Building Real-Time Data Systems by Example📖 @ 📅 Kafka Summit Europe 2021 📆 Raffle: https://twitter.com/bsak/status/1395095120450568195?s=21!

Looking forward to reading this highly anticipated book!

Data Streaming IT in italiano: @kafkastreamingit

#ApacheKafka #Kafka #KafkaSummit #ksqlDB #KStreams #streamprocessing #datastreamingit

Apache Spark ⭐️

DATA + AI Summit 2021 📅 24-28 MAY 📆

Free General Admission: https://databricks.com/dataaisummit/north-america-2021.

Data Streaming IT in italiano: @kafkastreamingit

#ApacheSpark #Spark #databricks #MLFlow #AI #Python #SQL #datastreamingit

ksqlDB 0.18.0 🚀 is out!

New features:
- Materialize Table-Table join results;
- support DEFINE and UNDEFINE statements in migrations tool
and much more.

See: https://twitter.com/ksqldb/status/1397600801581789185?s=21.

Data StreamingI T in italiano: @kafkastreamingit

#ApacheKafka #Kafka #ksqlDB #databases #streamprocessing #datastreamingit

Apache NiFi 💧

Cloudera Enters into Definitive Agreement to be Acquired by Clayton, Dubilier & Rice and KKR for $5.3 Billion

Cloudera Press Release: https://investors.cloudera.com/news-and-events/news/press-release-details/2021/Cloudera-Enters-into-Definitive-Agreement-to-be-Acquired-by-Clayton-Dubilier--Rice-and-KKR-for-5.3-Billion/default.aspx.

DataStreamingIT in italiano: @kafkastreamingit

#ApacheNiFi #NiFi #Cloudera #BusinessNews #streamprocessing #datastreamingit

Kafka Summit APAC 2021 📅 27-28 JUL 📅

Free registration: kafka-summit.org.

Data Streaming IT in italiano: @kafkastreamingit

#ApacheKafka #Kafka #KafkaSummit #ksqlDB #KStreams #KafkaStreams #streamprocessing #datastreamingit

What is (Apache Kafka based) Redpanda 🔴🐼? 🤔

Find out in this episode of Software at Scale podcast 🎧: https://www.softwareatscale.dev/p/software-at-scale-18-alexander-gallego.

DataStreamingIT in italiano: @kafkastreamingit

#ApacheKafka #Kafka #Redpanda #streamingplatform #datastreamingit