In the system design diagram there are:
- 3 chats,
- every chat has 4 users.
- GW -> socket gateway server that handles client's socket.
- Chat API is separate for simplicity to not cover processing of the messages on the GW itself

You can see that if we have much larger participants count than number of GW nodes we end up every GW node handle ALL large chats, thus causing broadcast.

===

This is one of the canonical solution that I saw on the internet for Messenger: you connect user to random GW and this GW gets (somehow) the message to deliver to the users that are connected to it. The solution is actually the way it works in Telegram and other Messengers, you have only single socket connection, it does not change when you switch chat.

First screen: 2 GW, 4 users per large chat => 4 / 2 = 2 users on EACH GW for EACH large chat
Second screen: we scaled GW nodes by 2 = 4 GW nodes -> and it didn't fix the broadcast, thus non linearly scalable

Other way to show it. In this diagram (screen 1), we have only 1 chat with 4 participants per chat and 1 message per second coming to this chat from one of the clients (out of scope).

Do you see it? Every WS Gateway have to process all messages, as chat participants are connecting randomly (or deterministically but based on user_id property).

From now on, unless number of WS Gateways is more than max number of participants in chat - it will always be the case. Check screen 2, number of nodes = 5 > participants per chat = 3 (top 3 clients => chat1, bottom 3 clients => chat2). Now WS gateways process 1 message per second, except WS Gateway1, because he has users from both chats.

Screen 3 - is the same scaled situation but with single chat