RCON Best Practices

When developing an RCON protocol, there are three main options:

• Source RCON protocol, which is commonly used.
  • Source protocol is also the protocol where we encounter the most issues.
• Websockets.
  • Years ago the Rust game developers switched from Source protocol to a custom Websocket-based RCON which we've found much easier to work with.
• The BattlEye protocol used by Arma and DayZ.

Any of these protocols are perfectly fine when implemented correctly. We recommend going with whatever is easiest for you to use. The commands and information you make available are far more important for improving server administration.

Some of the pros/cons we've seen when working with these protocols are outlined below.

Source

There are many client implementations available for Source. We generally use and recommend mcrcon for quick connection checks and testing.

This is the closest thing we have been able to find for a protocol specification: https://developer.valvesoftware.com/wiki/Source_RCON_Protocol

Despite being a fairly standard protocol, most games that use Source protocol will have game-specific clients that make better use of available commands and work around any implementation-specific bugs.

We work with many Source protocol implementations and have found many issues to work around. Most of these issues are implementation bugs and oversights and not an issue with the protocol itself.

These are the most common problems that we see:

1. Character encoding issues. The Valve wiki calls for ASCII. We highly recommend breaking with the spec on this and using UTF8. Multiple times we've seen a developer use an ASCII implementation, but send UTF8. This will cause problems and generally break the protocol due to UTF8's variable length encoding (i.e. the string length is not necessarily the byte length). Test your implementation with UTF8. Try adding emoji or unusual characters to your player name in testing.
2. A response packet is limited to 4096 bytes. Responses can be split, but we see issues with this frequently. After the first packet we'll often see missing headers, data arriving out of order, parts of the previous command arriving with the next, etc.
3. Many of the Source games we support can not handle multiple commands at the same time. When we send multiple commands we'll see some get ignored, get broken responses, etc. This is mostly a problem in combination with the above and is not any inherit limitation of the protocol, just buggy implementations of it.

4. There is no indication for the end of a response. There is a workaround that involves sending a SERVERDATA_RESPONSE_VALUE  packet after every command. When you receive a response to the SERVERDATA_RESPONSE_VALUE  packet you should have the full command response.

   The problem here is many games don't implement this behavior and ignore and/or throw an error when this is done. For those games we resort to sending a fast/simple command instead which incurs the penalty of a larger and unnecessary response.

5. The protocol does not specify a method for emitting events. Several games we support send events as a standard packet with a custom packet type. Generally we see the packet type set to 1  or 4  as they're unused. This is our preferred method and avoids several issues we see with other games.

   Another workaround we commonly see for this limitation is a command to retrieve chat/events. Usually the command returns the events since the last time the command was run and then clears the buffer/queue. This is slow and effectively limits you to one RCON client since the buffer gets wiped each time.

Websockets

Websocket server and client specifications are well defined by the IETF and there are implementations available in just about every language out there. Even web browsers can use Websockets.

The only games I know of using Websockets currently are Rust and Battlebit (implementation written by us, repository here: https://gitlab.com/BattleMetrics/battlebit-rcon).

A small amount of data with the messages will be required to help identify commands and responses.

As an example a command and response could look something like the below examples (JSON encoded). It could really be anything if something else makes more sense for your game. The only thing strictly necessary is a way to identify what message goes with which command.

Request:

{
  "type": "command",
  "identifier": 123,
  "payload": "status"
}

Response:

{
  "type": "command-response",
  "identifier": 123,
  "payload": [
    {
      "steamid": "11111111111111111",
      "name": "player",
      "score": 1337,
      ...
    }
  ]
}

BattlEye RCON

Arma 2/3 and DayZ are the only games we've seen that use BattlEye RCON protocol. We've found it to be reliable and easy to work with. The specification is simple, easy to understand, and there are multiple clients available that work well. We usually recommend using BattlEye RCON console when diagnosing issues with our users.

There are only two downsides we've found with BattlEye RCON:

1. The main limitation is the 1 byte sequence number. This limits you to a maximum of 255 response packets which is frequently hit by communities with large ban lists.
2. The protocol is UDP based. It takes us longer to detect dropped connections and re-establish connections when a server shuts down/crashes/goes away. It also makes it harder to diagnose connection issues. With a TCP based protocol we may receive connection refused errors which tells us the firewall is open and rules out some problems.

The protocol specification is available on the BattlEye website here:

https://www.battleye.com/downloads/BERConProtocol.txt

General implementation details

With a kick and player list command we can implement most BattleMetrics functionality. A way of receiving chat and server events is also greatly appreciated by both us and server admins.

The following is basically a wishlist for what we'd like to see from games. The games that provide even half of these features are few and far between. Not a single game we support has them all. Having an RCON protocol at all will put you ahead of most games.

• Any data associated with a player (chat message, kill, join/leave event, etc.) should have a unique identifier such as a Steam ID.

  Many of the games we support only include a player name with chat messages. This makes it impossible to determine which player sent a message when multiple players have the same name. Players also use invisible or zero width characters to make it harder to ban them.

• We'd recommend using a standard data format such as JSON that can handle escaping characters. It makes things easier for us and you. You can pretty print the output if you want to retain human readability.

  With the exception of a few Rust commands all games we support use plain text for commands and their responses. It's common, simple, and makes command line interfaces easy, but it also makes it possible for players to create names that break attempts at parsing.

  We most commonly see issues with this when commands take multiple parameters and/or require parameters to be in quotes without providing a means for escaping. As an example, we've seen games where you have to ban a player by name, but it's not possible to ban them if their name contains a space or quote.

• Commands that interact with players should use the platform ID (e.g. Steam ID) and not the player's name. This goes along with the previous two points. Platform IDs are generally safe and can be used anywhere. Player names are user input, are provided by potentially malicious users, and usually allow all types of characters.
• Most games provide some sort of built-in ban system. Some are much better than others. The best ones provide enough functionality that we're able to interface with the system to allow for pushing bans to multiple servers, sharing ban lists, etc.
• The best systems we've seen implement the following:
  • Allow banning of offline players. Quite a few games won't let you ban players that aren't currently online. This prevents us from pushing bans using our ban sync system and also prevents admins from banning hackers/trouble players after they've left the server.
  • Prevent banned players from connecting. Some games don't remove players until the player has loaded in. This can provide a brief period where the player can still cause issues.
  • List bans. It's useful to see who you've banned, especially if you're running multiple servers. We also use it to verify that bans have been added/removed when we sync them.
  • Durable bans. Bans don't get added and removed that often. Save them to disk when changes are made, not just at shutdown. Crashes happen.
  • Fast ban lookup. We see a lot of games that load a ban file and then iterate over it every time a player connects. Large communities end up with large ban files. This can quickly become a performance problem.
• A command to list players and associated information. This should at the very least include the player's name and platform id. Other information commonly include is the player's ip, ping, score, kills, team kills, and kit/loadout/gun.
• Events broadcast over the RCON connection allow us to react in near real time to player updates and game events. This includes things like player's joining and leaving, kills, chat messages, etc.
• Most games provide a way of sending messages to players. Common uses are answering questions, warning players, and announcing restarts. A simple command to send a chat message is a good start, but additional commands to privately message a player, send warnings to players that must be manually dismissed, and broadcasts that appear in a more prominent location are very useful for admins.
• Commands to mute and unmute players are always appreciated by admins. If your game has voice chat these commands can mute both systems, or they can be separated out.
• It is becoming more common for games to be sold in multiple stores (e.g. Steam, Epic, Microsoft). Each store usually comes with its own unique platform identifier. Some of these identifiers are difficult to distinguish from each other. If you have, or might have multiple type of identifiers you may want to include the identifier type where identifiers are used.
• If a command has a set number of valid values another command to retrieve those values is very helpful. An example of this would be change and list maps commands.
• A command or document should be included that lists what commands are available and what if any input they take. This is frequently provided by a help  command.

BattleMetrics Implementation

Once a game has functioning RCON the main thing we need is documentation. Documentation on how to start a server and configure it are frequently left to the community to divine. Information on what RCON protocol was used is also appreciated. As stated earlier, documentation on RCON commands can be provided by a command, or a wiki, it just needs to exist and be accessible.

Most games are unique and developers will implement the commands that make sense for their games and communities. As a result we always expect to spend time adapting to and interfacing with each game.

When we have a server running with RCON we'll generally join and try to connect to RCON. Once we're connected we can start the process of adding the game. Commands are run to verify their inputs and outputs and implement parsing as necessary. We can then start work on dialogs for commands, configuration for triggers, and other various integrations. This usually doesn't take more than a day or two, but it varies with the complexity of the game and number of commands and event types.