Skip to content

Security of the ports a node opens

This transport is designed for a trusted network between services you deploy. It is not designed to face the public internet.

That sentence is the page. The rest is what the library does to hold you to it, and what it cannot do for you. A node opens at most two ports, and the sentence covers both of them. The remote transport is everything up to the last section. The management port is the last section.

Opening a port that accepts frames naming actor paths and message types is a serious surface, so the defaults are set for somebody who has not thought about it yet.

Loopback by default, and a refusal rather than a risk

bind_host is 127.0.0.1. Binding to anything else without a secret raises InsecureRemoteConfig while the system is being constructed, naming both settings.

"Anything else" includes the empty string. It reads like a setting nobody filled in, which is exactly how it reaches a deployment, and the sockets layer reads it as every interface. So bind_host="" is 0.0.0.0 written differently, and it is refused the same way. Only 127.0.0.1, the other loopback literals, and localhost count as loopback.

A misconfigured deployment therefore fails to start instead of quietly serving strangers. It happens during construction, before the port is listening and before any ref has been handed out, so there is no window in which the wrong configuration is running.

The handshake

Before any application frame is read, both sides exchange the system name, the canonical address, a system uid, the wire protocol version, the tapio version, and a proof that they hold the shared secret. A bad proof or a protocol mismatch closes the connection with a logged reason, and nothing further is read.

Each proof covers the whole exchange. It is an HMAC, under the shared secret, of which side wrote it, both sides' nonces, and the identity of each side that has named itself so far: the system name, address, uid and release. A proof of a nonce alone would make every node a signing oracle. Anything a node can be made to dial, through a ref in a message, a seed, a DNS entry or a member address in gossip, could forward a guarded system's challenge to it and pass the answer on under an identity of its choosing. Bound to the exchange, an answer is good only for the identity written beside it, so changing who is speaking fails the proof, and so does reusing one side's proof as the other's.

The side that was dialled says almost nothing first. Its opening frame is a challenge and the protocol number. The name, address, incarnation and release travel in the welcome, which is written only once the dialler has answered that challenge. A listening port answers anything that can reach it, so a first frame carrying those four would hand a scanner the deployment's identity and the exact release it runs for the cost of one connection.

A dialled system that will not use the link answers with a declined frame in place of the welcome. It carries the reason and nothing about the system that sent it, and it too is written only after the dialler has answered the challenge. The two reasons are a refusal, such as a quarantine, and a simultaneous dial that the other link wins. Saying so before the welcome is what keeps the dialler from writing into a link that is about to be closed.

The dialler still names itself in its answer, so this is not symmetrical. It is not the same exposure either: a dialler chose the address it dialled, where a listener chose nobody. Closing the other half would need a fourth frame and a second round trip on every connection.

Protocol equality is required rather than negotiated. An incompatible wire format that half works is worse than one that refuses.

The tapio version is not checked, only reported. It moves for a fixed docstring and a faster mailbox, neither of which a peer can observe, so pinning a link to it would make every release a flag day: during a rolling deploy half the nodes would refuse the other half. Two nodes on different releases talk to each other as long as neither release changed the wire. What they cannot do is disagree about the protocol.

The system uid is minted per system incarnation, and it is what makes a restarted peer a different peer rather than the same one returning. Actor uids are random too, so a ref held from before a peer restarted names no actor in the new incarnation, even one spawned under the same name. An association is bound to the uid it handshook with, so a peer that reconnects presenting a new uid means the old one died: the previous association is quarantined and its watchers are told, instead of the new connection silently inheriting the old one's identity.

The secret authenticates, TLS is what makes it private

secret proves that a peer knows something a stranger does not. It does not encrypt anything. Over plaintext, the shared secret protects the handshake and nothing else: every frame after it is readable and modifiable by anything on the path.

The secret also cannot tell a relay from a direct connection. A party with no secret that a node can be made to dial can still pass an exchange through unchanged, and so connect two honest systems under their real identities with itself in the middle. It cannot change either identity, but every frame after the handshake passes through it. Only TLS prevents that, because it authenticates the connection itself rather than what is said over it.

tls takes a certificate, a key, and an optional CA for mutual authentication. Both ends of a link, the management port and the tapio-cluster command require TLS 1.2 or later, whatever the Python build's default. Use both for anything crossing a machine boundary. The secret without TLS is only appropriate where the network itself is the boundary, such as a container network or a loopback interface.

The certificate and key are loaded while the system is being constructed. A file that is missing or cannot be read makes ActorSystem(...) raise OSError and release the port. It does not leave a port bound that never answers.

The decoder is the trust boundary

Everything a peer sends arrives at one place, and that place is written like the boundary it is:

  • A frame size cap before allocation. Frames over max_frame_bytes, four megabytes by default, are refused without reading them, which is the cheapest way to stop a hostile or buggy peer from exhausting memory.
  • A smaller cap before authentication. A peer that has not finished the handshake has proved nothing, so a handshake frame over 4096 bytes is refused, whatever max_frame_bytes says. A real hello is a few hundred bytes.
  • A bound on how many peers can be handshaking. At most max_pending_handshakes inbound connections, 128 by default, are handshaking at once. A connection past that is closed as it is accepted. Each one has handshake_timeout to finish, and on a TLS listener that deadline covers the TLS handshake too.
  • Registry-only type resolution. The t field of a frame is a registry key, never an import path. Resolving a dotted name from a socket into an importable object is remote code execution, and it is how this goes wrong casually. An unregistered key becomes a dead letter naming the key, and nothing is imported to find out what it might have meant.
  • Validation as the decode. A frame becomes a message by being validated against the registered model, in Pydantic's default lax mode. A missing field, or a value that cannot be converted to the field's type, is a decoding failure and not a surprise inside a handler. A value that can be converted, such as "5" for an int, is converted. Set strict=True in a message's model_config to refuse those too.
  • No reflection anywhere in the path. Nothing a peer sends selects code to run. It selects a registered model to validate against, and that is all.

An actor path in a frame is likewise only ever looked up in the local registry. A path naming an actor that does not exist is a dead letter, so a peer can discover nothing by probing except which of its own guesses failed.

What is left to you

A peer that passes the handshake can send any registered message to any actor whose path it knows. There is no per-actor authorization, and there are no capabilities beyond the ones you build: the shared secret is the whole authorization model. Everything behind it is one trust domain.

The one place the library gives you a lever is remote spawning. A spawner offers named factories and refuses everything else, and the allowlist, spawner(offers=["worker"]), is checked where the spawner is written.

An actor that will start anything registered, on request, is a capability handed to whoever can reach the port. Keep the list short, and keep it to things that are safe to have started by a peer.

The management port

A node with ManagementSettings opens a second port. An operator reads membership and downs a member through it. It is off unless it is configured, and it binds loopback by default. Binding it beyond loopback with nothing to authenticate an operator is refused, the same way an unsecured remoting bind is. The clustering page covers all of that.

This page covers the deployment the port expects: loopback, or behind a sidecar that terminates TLS and authenticates the caller. It is not written to be an internet-facing API. It answers one request per connection. It holds at most thirty-two connections at once and refuses the rest with a 503. It gives each request thirty seconds to arrive and be answered.

Over TLS the same limits cover the TLS handshake. The port accepts plain TCP and starts TLS on each connection itself, so a connection that never sends a hello counts against the cap and is closed at the deadline. A connection refused by the cap over TLS is closed with no 503, because an answer could only be sent after a handshake, and the handshake is the cost the cap bounds.

Those limits are there because this port shares an event loop with the cluster daemon. A flood that starves the loop delays gossip and heartbeat replies. The node then stops answering probes, its watchers call it unreachable, and a cluster with a downing strategy removes it. The cap keeps the cost of a flood fixed, and it keeps it deliberately small. Thirty-two connections is enough for a human, a script and a health probe at the same time. It is not enough to serve a crowd, so put a proxy in front of the port if that is what you have.

A checklist

For anything beyond one machine:

  • secret set, from the environment or a secret manager, never in the source.
  • tls configured, with mutual authentication where both ends are yours.
  • bind_host set to the interface you meant, not to 0.0.0.0 or "" because it was easier.
  • The port reachable only from the services that need it, at the network level, because tapio has no per-peer allowlist.
  • max_frame_bytes no larger than your largest legitimate message.
  • Spawner allowlists reviewed the way you would review a public endpoint, because that is what they are.
  • The management port, if one is open, on loopback or behind a sidecar, with a token or a client certificate the moment it is anywhere else.