Знаете этого парня, да?
- https://leanpub.com/u/johnbywater
А его книгу?
- https://leanpub.com/dddwithpython

#DDD #DistributedSystems #EIP #EDA #Microservices

Один из наиболее частых вопросов - есть ли жизнь без Outbox pattern?

Да, есть, и Outbox нужен далеко не всегда.

В руководстве Microsoft https://docs.microsoft.com/en-us/dotnet/architecture/microservices/multi-container-microservice-net-applications/subscribe-events#designing-atomicity-and-resiliency-when-publishing-to-the-event-bus
и в каталоге Chris Richardson
https://microservices.io/patterns/data/transactional-outbox.html
в качестве альтернативы Outbox приводится Event Sourcing, иногда реализуемый в виде Front-Door Queue
https://github.com/obsidiandynamics/goharvest/wiki/Comparison-of-messaging-patterns#front-door-queue . При этом, становится важной поддержка идемпотентности.

В EIP ("Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions" by Gregor Hohpe, Bobby Woolf) есть весьма познавательная, и, к сожалению, малоцитируемая глава "Transactional Client", которая рассматривает вопрос "Transactional Consumer" vs "Transactional Sender".

📝 "Both a sender and a receiver can be transactional. With a sender, the message isn’t “really” added to the channel until the sender commits the transaction. With a receiver, the message isn’t “really” removed from the channel until the receiver commits the transaction. A sender that uses explicit transactions can be used with a receiver that uses implicit transactions, and vise versa. A single channel might have a combination of implicitly and explicitly transactional senders; it could also have a combination of receivers.

With a transactional receiver, an application can receive a message without actually removing the message from the queue. At this point, if the application crashed, when it recovered, the message would still be on the queue; the message would not be lost. Having received the message, the application can then process it. Once the application is finished with the message and is certain it wants to consume it, the application commits the transaction, which (if successful) removes the message from the channel. At this point, if the application crashed, when it recovered, the message would no longer be on the channel, so the application had better truly be finished with the message.

How does controlling a messaging system’s transactions externally help an application coordinate several tasks? Here’s what the application would do in the scenarios described above:"
- "Enterprise Integration Patterns: Designing, Building, and Deploying Messaging Solutions" by Gregor Hohpe, Bobby Woolf

Этот же вопрос, но немного с другой терминологией, "Transactional Client" vs "Transactional Actor", рассматривается в RMPwtAM ("Reactive Messaging Patterns with the Actor Model: Applications and Integration in Scala and Akka" by Vaughn Vernon).

Те, кто пишут на Golang, могут использовать коробочные решения Outbox:
- https://github.com/ThreeDotsLabs/watermill/tree/master/_examples/real-world-examples/transactional-events
- https://github.com/obsidiandynamics/goharvest
- https://github.com/wework/grabbit (поддерживает Saga)
- https://github.com/omaskery/outboxen
- https://github.com/omaskery/outboxen-gorm
- https://github.com/QuangTung97/eventx
- https://github.com/nilorg/outbox

- https://github.com/topics/transactional-outbox?l=go
- https://github.com/topics/outbox?l=go
- https://github.com/topics/outbox-pattern?l=go
- https://github.com/topics/transactional-outbox-pattern?l=go

Watermill интересен тем, что данные в таблицах неизменяемы, т.е. записи ложатся компактно и постранично на диск, и могут быть применены решения для time-series data (timescaledb, pg_partman/pg_pathman...).

#DistributedSystems #EIP #EDA #DDD #Microservices #Golang #SoftwareArchitecture #SoftwareDesign

Ну и, чтобы уже закрыть тему Outbox, то стоит упомянуть несколько полезных ссылок по Transaction log tailing (Transaction log mining):

- https://www.postgresql.org/docs/11/sql-notify.html
- https://www.postgresql.org/docs/11/sql-createpublication.html
- https://www.postgresql.org/docs/11/protocol-replication.html
- https://tapoueh.org/blog/2018/07/postgresql-event-based-processing/
- https://github.com/eulerto/wal2json
- https://github.com/kingluo/pgcat
- https://github.com/hasura/pgdeltastream
- https://github.com/pramsey/pgsql-http

Игорь Кузнецов ( @swenotes / @kuznetsovin ) недавно писал статью на эту тему:
- https://www.swe-notes.ru/post/pg-event-sourcing/

Для Golang-разработчиков могут оказаться полезными следующие ссылки:

- https://pkg.go.dev/github.com/lib/pq#hdr-Notifications
- https://github.com/blind-oracle/psql-streamer
- https://github.com/jackc/pglogrepl
- https://github.com/dingchao/prometheus/blob/43953b105bf9dddd6ec82878f509fb3628e1961b/storage/remote/wal_watcher.go
- https://github.com/redesignhealth/postgres-wal-tailer/blob/2562429aa1feb692987bbb7775d0555ebbea43b2/src/main/java/jon/pgrepl/WalTailer.java#L43

#DistributedSystems #EIP #EDA #DDD #Microservices #Golang #SoftwareArchitecture #SoftwareDesign

Тут как раз Chris Richardson написал blog-post, затрагивающий проблему atomicity and resiliency of event publishing:
"Events on the outside, on the inside and at the core" by Chris Richardson
http://chrisrichardson.net/post/microservices/2021/02/21/events-are-the-core.html

Слайды заслуживают внимания. Также они демонстрируют сегодняшние тренды в микросервисной архитектуре, и какую роль в этом играет DDD.

#DistributedSystems #EIP #EDA #DDD #Microservices #Golang #SoftwareArchitecture #SoftwareDesign

Слайд 70 заслуживает отдельного поста. Варианты реализации OO/Functional Aggregates на примере Reference Applications by Chris Richardson:

Traditional OO mutable Domain Objects:
- https://github.com/cer/event-sourcing-examples/tree/master/java-spring

Functional Scala witn immutable Domain Objects:
- https://github.com/cer/event-sourcing-using-scala-typeclasses

Hybrid OO/Functional Scala with immutable Domain Objects:
- https://github.com/cer/event-sourcing-examples/tree/master/scala-spring

#DistributedSystems #EIP #EDA #DDD #Microservices #Golang #SoftwareArchitecture #SoftwareDesign

Greg Young пишет книгу "Versioning in an Event Sourced System":
- https://leanpub.com/esversioning

Можно прочитать online: https://leanpub.com/esversioning/read

Готова на 90%, правда, последнее обновление было 2017-04-10 (если верить сайту).

Посвящена популярному вопросу - что делать с логом событий Event Sourced агрегата, когда мы изменяем агрегат (добавляем, удаляем, изменяем его атрибуты).

Парень написал конспект по книге Greg Young "Versioning in an Event Sourced System":
- https://github.com/luque/Notes--Versioning-Event-Sourced-System

#SoftwareDesign #DDD #Microservices #EDA #CQRS #EventSourcing #SoftwareArchitecture

Парень написал конспект по книге Greg Young "Versioning in an Event Sourced System":
- https://github.com/luque/Notes--Versioning-Event-Sourced-System

#SoftwareDesign #DDD #Microservices #EDA #CQRS #EventSourcing #SoftwareArchitecture

Вышла неплохая статья о шифровании чувствительных данных в Event Sourcing:
"Protecting Sensitive Data in Event-Sourced Systems with Crypto Shredding"
- https://www.eventstore.com/blog/protecting-sensitive-data-in-event-sourced-systems-with-crypto-shredding-1

#SoftwareDesign #DDD #Microservices #EDA #CQRS #EventSourcing #SoftwareArchitecture

Пару лет назад проблему удаления персональных данных из неизменяемого Event Sourcing лога событий по требованию GDPR освещал Mathias Verraes, и предложил два подхода:

"Eventsourcing Patterns: Forgettable Payloads. Store the sensitive payload of an event in a separate store to control access and removal." by Mathias Verraes
- https://verraes.net/2019/05/eventsourcing-patterns-forgettable-payloads/

"Eventsourcing Patterns: Crypto-Shredding. Encrypt sensitive information in an event and delete the key." by Mathias Verraes
- https://verraes.net/2019/05/eventsourcing-patterns-throw-away-the-key/

#DDD #EventSourcing #Microservices #SoftwareArchitecture #SoftwareDesign

"Modelling Reactive Systems with Event Storming and Domain-Driven Design"
- https://blog.redelastic.com/corporate-arts-crafts-modelling-reactive-systems-with-event-storming-73c6236f5dd7

#DistributedSystems #DDD #Microservices #EDA

Process Manager (Orchestration-based Saga) implementation with Watermill (Golang)
- https://github.com/czeslavo/process-manager
- https://github.com/ThreeDotsLabs/watermill/issues/7#issuecomment-753450071

#DistributedSystems #EIP #EDA #DDD #Microservices #Golang #SoftwareArchitecture #SoftwareDesign