Routing messages to PIDs in Erlang

Question

I'm writing a server in erlang that handles very large amount of messages (records). Each message has a tag (atom) like a user id.

The "router" will spawn a dedicated-permanent process for this user (to accumulate messages for few minutes before saving them and passing them) if such process doesn't exists. Otherwise it will pass it to the existing process mailbox as message.

The problem is the bookkeeping of a routing table.

I could think of serializing the router, each message will result in ETS lookup to find a PID by userId and eventually Spawn and ETS insert if it didn't exit. But that got congested in few seconds.

An alternative is to spawn a process directly to route each message, but that might result in race condition if a few messages to a single user came in in close sucession and didn't find their corresponding PID in ETS and spawned permanent processes. Messages will be lost and only the last spawned process will be valid (overwrite others in ETS) where the others will go idle and untracked.

I also might be thinking all wrong. is there a better way to handle this scenario?

Answer 1

你的第一个解决方案听起来适合这种情况。对我来说，如果你有大量的消息，那么为每条消息启动一个过程听起来是不对的。 ETS速度快，应该能够处理大量数据。

您可能需要考虑使用erlang OTP gen_server。您可以详细了解here，here和here。

此外，如果您要从多个流程访问ETS，您可能需要考虑{read_concurrency, true}和{read_concurrency, true}选项。详细了解here和here。

Answer 2

每个用户拥有一个进程（一个服务器）是一种常见的模式，就像你建议的那样。

有时，如果使用的协议允许它，而不是将消息路由到＆＃34;用户＆＃34;从服务器监听所有用户的过程，在任何连接之前产生新进程，监听（等待）新用户连接请求。当请求到达时，将产生新的等待进程，并且当前进程将管理与新用户的完整会话（有关详细示例，请参阅learyousomeerlang: a bucket of socket）。