Netcode

Netcode is a blanket term most commonly used by gamers relating to networking in online games. Players often infer "bad netcodes" when they encounter connection problems in a game. Some common causes: high latency between host and client, packet loss, network congestion and external factors independent to network quality such as frame rendering time or inconsistent frame rates.

Libraries
Libraries are sets of code meant to be shared between different programs. They are used for various games.

Pia
At its core, and  uses a library called "pia". Pia was developed by Nintendo to make creating networking features for games easier. It implements network features based on a "peer-to-peer" architecture, unlike the traditional “client-server” architecture. This means that instead of one server having a "one-on-one conversation" between each Nintendo Switch console to exchange information, all of the consoles communicate directly with each other to share information.

Pia assigns one console as the "host" of the session. This is usually the person who opened the session first, and people who join the session are considered to be regular clients. (Contrary to popular belief, it hasn’t been definitively proven that the person at the top of the matchmaking screen is the host of the session.)

NEX
NEX is a two-part system comprising a game server and an API used to communicate with the servers. NEX handles matchmaking for peer-to-peer sessions, NAT traversal (helping people behind a firewall or router with getting connected to others), and rankings. Pia has built-in support for NEX's matchmaking and NAT traversal features, making it easy for developers who are already using pia to implement online play.

The NEX system is a variant of Quazal Technologies Inc.’s Rendez-Vous online middleware, which was developed as far back as 2003. Quazal was purchased by Ubisoft in 2010, who then either licensed or sold the Rendez-Vous system to Nintendo.

NPLN
NPLN is a new multiplayer server system planned to replace NEX and be used for.

Clones
Splatoon 2 netcode is built entirely using a feature of pia called "clones". A clone is created by a console, and other consoles can "subscribe" to the clone. If the owning console makes changes to the clone, the subscribing consoles are notified of any changes. For example, a clone can be made using a variable called "X". If the owning console changes X to 5, then all of the subscribers will be notified that the value of X was changed to 5. Splatoon 2 does not use any other method of data synchronization other than clones, though pia supports sending messages directly between consoles without using clones.

There are three types of clones:


 * Unreliable – changing the clone will cause a notification to be sent to subscribing consoles, but the notification may or may not arrive at the subscribers because of packet loss
 * Reliable – a notification will be sent to subscribing consoles, with more being sent if they are not received, guaranteeing synchronization
 * Event – clone synchronization is guaranteed, and any data changes in rapid succession are guaranteed to be processed in the order that they were made

Here are some examples of where the various clone types are used:
 * Unreliable – player position, as the game can interpolate (guess) where the player is heading towards based on their current speed and position if there is data loss
 * Reliable – basic player information like their name and equipment, as this information is required to be synchronized between consoles, but the order in which it is received does not matter
 * Event – a boss Salmonid spawning in Salmon Run, as other consoles need to know that this happened and the order in which bosses spawn in

In addition, clones have various "access permissions" that can be set:


 * Send – values can be only set and sent to subscribers with an associated Receive clone


 * Receive – values can be only received from an associated Send clone
 * Sequential – all consoles can set the values, though because of this an Event clone cannot also be a Sequential clone (impossible to preserve the order of values set)
 * Atomic – all consoles can set the values, but only one console can set the value at a time

To keep synchronization, each console starts up a "clone clock". For Splatoon 2, this clock goes up 120 units per second and continues until the session closes. (For the nerds, an integer overflow wouldn’t happen for a very, very long time. It would be over a year before it overflows assuming that the data type is an unsigned integer.) If the clocks become out of sync, the clocks are resynchronized. This can occur if, for example, there is a sudden spike in CPU load and the game can’t keep up with its refresh rate. This clock value is also occasionally used as a “seed” for the random number generator (RNG), which is used in various in-game actions that involve randomness.

Tick rate
The tick rate situation in Splatoon 2 is complicated and marred with controversy. The controversy part mainly comes from a post by Octobyte, who published an article saying that Splatoon 2 runs at 15.75Hz. This isn’t entirely true.

To start, the game ticks at 60Hz. (Fun fact – when the game enters the plaza and halves the frame rate, it accomplishes this by lowering the entire tick rate of the game to 30Hz.) With each tick, the game tells pia to send packets, receive packets, and process any events. This means that the netcode also processes at 60Hz. However, Nintendo set a configuration flag that says clone data should only be sent every 4 ticks. This means that the tick rate for sending data is artificially constrained to 15Hz. Normal processing of packets occurs at 60Hz, so receiving and processing clone data runs at “normal speed”.

What does this 15Hz send rate mean?

The pia library will combine packets during the 4 tick waiting period into one big packet at the end. (This coalescing of data explains why Octobyte found an increase in the average packet size compared to Splatoon.) This waiting period also means that data may arrive a minimum of 4 ticks later. However, remember that event clones will always be processed in the order that they occur. Event clones are used for things like player damage, inking the map, and bullet spawning. This may mitigate some of the effects caused by the lower tick rate.

Why is Nintendo artificially constraining the tick rate?

This is likely for bandwidth reasons, as Octobyte stated. According to her data, there was a 45% reduction in bandwidth going from Splatoon to Splatoon 2. It could also be for stability reasons. By using less data, it can allow players with less capable Internet connections to be able to play online. This is especially beneficial for mobile hotspots.

How does this compare to Splatoon?

Unfortunately, it’s very hard to analyze Splatoon netcode. Nintendo made it easy for me to analyze Splatoon 2 since they accidentally left debugging information for many versions. Splatoon, on the other hand, is essentially a black box. I have found a configuration setting that may be similar to the “waiting period” setting in Splatoon 2, but I have been unable to confirm this. In addition, Splatoon uses a very old version of pia, and there have been several major changes to the library since (mostly relating to how packets are serialized). It would be unfair to do a “direct” comparison between Splatoon and Splatoon 2.

However, I do want to say that I believe Octobyte’s claim that Splatoon had trouble keeping up at 25Hz is incorrect. The fluctuation in data size could be there for other reasons. For example, the waiting period may be set lower, causing clone data to not be coalesced as often. This would result in lower overall packet sizes and data being sent much more frequently.

Disconnects
The game has three utilities to detect connection issues in-game.



What makes the game decide when to show the unstable message is that on every frame, the game checks if there is a "period of silence". A period of silence is an unusual gap in the usual communications where nothing is being received from other consoles. If one is detected, the game considers the connection to be "unstable". After around 5 seconds, the game gives up and brings up the "communication error has occurred" message. (If packets from other consoles are received again, the countdown is reset.) Pia could also choose to disconnect from the session if it considers the connection to be unrecoverable. There can be many reasons for a period of silence. It could be a momentary burst of lag on a person's Switch. There could be packet loss somewhere along the connection path. Someone could accidentally open the HOME menu and cause the game to go into the background. Because of this, it is difficult to know what exactly is the cause of every "the connection is unstable" message.

The second utility that the game uses is slightly more technical. There is a "clone clock" that synchronizes between all consoles. The clone clock is used with Event clones to ensure that they are processed in the correct order. Event clones will also wait for the receiving console to acknowledge that they have received the values before proceeding with sending additional updates. If there is a backlog of Event clones waiting for acknowledgements, the game will check each Event clone’s attached clock value to see when the data was initially sent. If the clock is 4 seconds or more in the past, the game will designate the receiving console to be a "bottleneck". If an Event clone is stuck waiting for an acknowledgement for that long, the receiving console likely has high latency or high packet loss on their connection. The console is then at risk of being kicked out of the session by force to maintain responsiveness, stability, and synchronization.

The last utility is that the game will check on every frame if the player is idle using based on various factors. These include current velocity, gyroscope movement, right stick movement, and more. If the player is found to be idle, a 60-second countdown is started. The countdown is reset to 0 if the game detects movement. If the player is idle for 60 seconds, then the game will bring up the "communication error has occurred" message and kick them out of the session. A penalty (temporary ban) will then be assigned to them.

What happens if the player loses connection to the host, however? Pia has a feature called "host migration" to solve that problem. When the session loses connection to the host, pia will consider all of the remaining consoles in the session as a candidate for a new host. The highest-ranking console will be selected as the new host and the session continues as normal. If the session is unable to migrate to the new host, then it will go to the next candidate in the ranking. This will continue until a host is chosen.

Matchmaking
There's a configuration file that seemed to indicate "lobby ID" numbers. For example, Regular Battle in Japan is set to ID 1. From the game's launch date to September, Regular Battle in North America and Europe was set to ID 1, but Nintendo changed it to 51 earlier this month. This led me to believe that the lobbies were segregated and people from Japan couldn’t be randomly matched with people in North America or Europe. I was then corrected by my followers on Twitter, as they were still getting matched with players in Japan. Further reverse engineering of the game revealed that my "lobby IDs" interpretation was incorrect. However, I was at least semi-correct when I assumed they were ID numbers.

Each VS multiplayer mode in Splatoon 2 has a specific "index number" assigned to it. These indices differ between regions.

Each number represents a specific matchmaking configuration. While everyone is put into the same matchmaking pool (per mode), the NEX game server will attempt to match a player using the configuration for their region and mode. To determine which session someone should be matched with, each criterion in the configuration is assigned a score value, and the session with the highest total score is chosen.

Here is the list of criteria that Nintendo can assign scores:
 * The difference in Power between the session owner and the joining player
 * The difference in the rate of disconnection between the owner and joining player
 * How far the distance between the owner and the joining player is (calculated using GeoIP)
 * If the country of origin matches
 * How long the session has been waiting for players
 * and more…

Because configuration data differs between regions and modes, Nintendo can adjust matchmaking in various ways. For example, for Japanese copies of the game, Nintendo might adjust Turf War matchmaking to prefer players within Japan. However, they might increase the threshold of the distance criteria when playing in Rank X lobbies to prevent long matchmaking times, as only 1% of the player base is in Rank X. In North America and Europe, Nintendo would also increase the threshold for distance in all modes as the distance between players can be higher overall compared to Japan.

The specific values used for the matchmaking criteria are only visible to Nintendo.