Netwerking Tutorial Request

[Shard]

I would like to request a Netwerking Tutorial (video/pdf) that shows how to use the basics of Leadwerks Netwerking.

 

After having looked through the wiki, I have several questions and am a little confused about how things are supposed to be done.

[AggrorJorn]

+1

[Drew_Benton]

The Netwerks library is a wrapper for ENet, which is a low level UDP networking library. Basically, it's good for simple tech demos or really simple games that just need to pass data around with minimal networking requirements and no real need for higher level interactivity. If you needed higher level features, such as lobby systems, auto patching, object replication and state management, you'd have to code all those yourself. If you are going to use this library through Netwerks or standalone, I'd suggest taking a look through the source to understand the internals better.

 

I've made a little demo showing the main uses of the Netwerks API. However, I can't recommend using it for your project because of the alternatives that are available. RakNet is pretty much the go to guy for higher level UDP networking and has a pretty indy friendly license:

Commercial Single Application License

 

Allows use of RakNet for one or more SKUs with any distrubtion method. It is acceptable to use the free Indy license until you first gross $250K. Discounts available for multiple platforms. Included with the Single Application license are free forum, email, phone, support. Standard support is free, with support contracts and custom engineering also available. Review the license online.

 

basically, once you make over 250k gross, then you need to buy a license for 5k per sku. Until then, you can use it for free so it's a win-win.

 

Back to Netwerks, I don't have the setup to make a video or PDF, so here's code that's commented that shows the main uses.

 

Server: http://drewbenton.pastebin.com/x8MjCU1D

Client: http://drewbenton.pastebin.com/F5DY37F9

 

Sometimes the demos crash on exit, so that leads me to believe there are some memory management issues between the DLL and the EXE. I've tried all combinations of not calling the FreePacket function and it still does it. I'm not sure how different the wrapper for ENet is than to the ENet API itself so Josh would need to have a look at that. Honestly though, I'd suggest Josh just rip out Netwerks from the engine. In the past, there was a lot of pressure by people to have networking built in, but it's just not practical in its current form.

 

Network programming is really tricky in that you have to make sure errors in the system don't bring down your entire application or corrupt data. There is a lot of theory in this, so my suggestion is to use a higher level networking library to take care of these things so you don't have to yet. Right now, I am learning NetDog because it is designed to handle certain things that you'd have to otherwise code yourself, all of which are not trivial. I can't comment on if it's worth licensing yet, but after I get some more work done with it I'll make a post about it.

 

Anyways, when using Netwerks, the basic of all your operations revolves around defining packets and then processing those packets. In my example, I used a 1 byte field for the opcode, but realistically, you might end up using at least 2 bytes to support over 256 messages if your game is that complex. I also added a very simple object replication system that stores events and then sends them back to the newly connecting clients. You have to account for a lot of other things for such a system, so it's not as trivial as it's shown there. You have to ensure objects still exist, send state information if they are moving, and so on.

 

I'd definitely suggest checking out RakNet for your project though. In the past, when RakNet was not free to use at all, suggesting it would have been out of the question, but now that it is, you really can't beat it. I'd only suggest using ENet from source and only if you already have experience in the networking domain since you will more than likely make some changes to the core library as you go along. When I last used it for a tech demo in trying out IOCP w/ UDP, I noticed how it'd be possible for a malicious client to crash the server by sending packets of large lengths over and over since ENet supports any sized packets through fragmenting.

 

I've done a bit of networking stuff the past couple of years, so if you have any questions feel free to ask!

[Shard]

I've done a bit of networking stuff the past couple of years, so if you have any questions feel free to ask!

 

 

Thanks for the reply.

 

I do have a couple of questions:

1) I am thinking about building an RTS game. I'm not a hundred percent sure, but that sounds like it would require a lot of data to be sent around. Would RakNet be good for this purpose? From the sounds of what you said, Netwerks would not be good for this at all.

2) Is a host-client the only way to go? Or can peer2peer also be done? Is one better than the other? How about in a networked game of up to six PC's?

3) How exactly do networked games work? How does one side keep track of the player on the other side or the ammo a player has or a the damage a players bullet does? I know that packets are "data", but what defines a "message"?

4) What is object replication, state management and patching (if its not patching in the traditional sense)

 

I will definitely look into RakNet.

[Marleys Ghost]

I was wondering what had happened to the leadwerks netwerks that was being tested a while back?

[Guest Red Ocktober]

yeah... me too... where'd that evaporate to...

 

--Mike

[Drew_Benton]

1) I am thinking about building an RTS game. I'm not a hundred percent sure, but that sounds like it would require a lot of data to be sent around. Would RakNet be good for this purpose? From the sounds of what you said, Netwerks would not be good for this at all.

 

The article, 1500 Archers on a 28.8: Network Programming in Age of Empires and Beyond, is one of the main references to use for RTS network programming to get an idea of the complexity involved. In addition, What every programmer needs to know about game networking, also gives some additional information regarding the topic. The networking model you would use for a RTS would be a "deterministic peer-to-peer lockstep networking model".

 

The task of coming up with a deterministic simulation, regardless of game genre is no easy task. As soon as you use floating point types in your calculations or have say a physics system that uses floating points, determinism goes out the window. For such matters as those, you'd want to read up on Floating Point Determinism, which should give you a good idea of the things you need to keep in mind when implementing your logic. You can count on existing physics middleware not being deterministic by default, so you have to resort to using the fancy physics API to being graphic eye candy and implement your own system on top of it that you can guarantee to be deterministic if you need physics to be a core part of your RTS.

 

Gamedev.net has some good threads to read up on comments about RTS network programming as well. Do a few searches there to get some additional ideas. From a thread there:

Original post by hplus0603

Peer-to-peer means significantly higher messaging overhead. Is there any reason why you wouldn't just tunnel all the data through the host?

 

Also, for RTS networking, it's customary to only use TCP, because there's too many units to work well over lossy UDP (check out the lock-step method, and the 1,500 archers article in the <a href="http://www.gamedev.net/community/forums/showfaq.asp?forum_id=15">Forum FAQ</a>).

 

I think the problem would be better solved with TCP since you need reliable, in order packets that are not going to be sent as a fast as the input would go in say a FPS game. The benefits of UDP are best seen when you can discard earlier packets in favor of more recent packets that are received, but such a model is not applicable in a RTS since everything has to be deterministic if you don't want the game to fall apart. You could make it work with UDP though, but it just depends on the final design of the game and how much data you are sending, which also affects how the TCP model would work as well. There's no easy answer here as compared with other game genres, but I'd just start with one that is easier for you to use and just try to make it to work.

 

With that said, I'd suggest using RakNet over Netwerks if you are going with UDP for now. If you are trying to make a game and have not ever worked with TCP before, then I'd suggest sticking to an existing library rather than trying to write your own because of how much can go wrong. Existing libraries like RakNet are tried and tested and your current goal is to write a game, not a networking library. There are low level TCP supported libraries around, such as boost::asio or POCO, but I'd advise getting started with these for the same reason to avoid ENet/Netwerks: you want something higher level so you can focus on your game rather than the networking library.

 

If you do have the time to spend to learn networking stuff though or it interests you, then by all means, try whatever you like. I ended up doing that and spent a while learning low level TCP and UDP concepts because I wanted to develop my own library rather than not understand what was going on under the hood of existing libraries. However, you will lose a lot of time and productivity, so I'd only recommend it if you aren't working on a main project.

 

2) Is a host-client the only way to go? Or can peer2peer also be done? Is one better than the other? How about in a networked game of up to six PC's?

 

This is explained a bit in the earlier articles, but you'd have a central server for lobby, chat, match making and then have a client/server model where the 'server' is the peer host of the game. A P2P model can work, but if you notice in the 1500 archers article, it was programmed by Ensemble Studios, whom I'd safely say had more resources and people than any of us have access to. I think if you just stick to the Client/Server model of peers in the game, you'd be able to have less to deal with when it comes to managing connections.

 

3) How exactly do networked games work? How does one side keep track of the player on the other side or the ammo a player has or a the damage a players bullet does? I know that packets are "data", but what defines a "message"?

 

How exactly do networked games work?

 

They work the same way a non-networked game works, with the exception a networked game has to process "messages" from external sources that occurred in the past (lag). A non-networked game can be thought of a specialization of a networked game where there is 0 network latency, since all events are processed immediately. You can easily simulate this as well. All you have to do is write a simple non-networked demo where you buffer key input by some delay (to simulate the ping to a server) and then after said time has elapsed, process those key inputs. You will notice the higher the delay is, the more inaccurate the simulation becomes. This is where Client Side Prediction comes into play as well as interpolation/extrapolation (see Entity Position Interpolation Code and Dead reckoning for example).

 

How does one side keep track of the player on the other side or the ammo a player has or a the damage a players bullet does?

 

Simply through the use of messages. In a client/authoritative server architecture, the server is the referee is validating events before passing them on to other clients. So a client connects to the server and the server creates a player object. This event is sent to other connected clients with the basic information of position, model, etc... If a client sends the event to shoot their BFG, it's the server's job to check to see if the player has that weapon first, and then if it has enough ammo to fire. If it does, then the server sends out the event. If it doesn't, the client gets a message that they are out of ammo and need to correct their game state with the server state. Object replication and state management are the key concepts here for such interactions.

 

I know that packets are "data", but what defines a "message"?

 

The programmer, you, defines network messages to mean certain things. Then, the client and server process those messages and update their simulation of the game world based on it, taking into account the time the event occurred. The easiest way to think about it is like the Win32 event model. When you get a message in your Window proc, you know how to process the message based on the specifications Microsoft has given you. So when you get a WM_KEYDOWN message, you know exactly how to use the wParam and lParam fields. The identifier WM_KEYDOWN is the opcode for the message and then the data is stored in two 32bit types.

 

For your own networking messages, you might choose to have a byte (256 unique), word (65k unique), or dword (4bln unique) opcode and then based on that opcode, you define the structure for the message. This is what defines your networking protocol. This protocol is the same type of protocol that you are already using when you use TCP or UDP. In either of those links, you'll see the format of the underlying packets that are sent across that protocol.

 

So, let's say you are using UDP and your packet protocol is as follows:

Header

Opcode [2 bytes]

Body

Data [0..1022 bytes]

 

Let's say you assign opcode 0 to be an operation where the client sends their name. Let's say this is your first time designing packets, so you implement it like you would in a regular program. The higher level packet structure for opcode 0 would look like:

 

word opcode

int nameLength

char * name

 

To process the packet, your logic would look something like:

void ParseOpcode0()
{
  nameLength = ReadInt();
  std::string tmpName;
  tmpName.resize(nameLength);
  ReadArray(&tmpName[0], nameLength);
}

 

Looks harmless enough, right? The golden rule of network programming is to never trust client data. Many professional commercial companies forget this rule all the time and pay as a result when hackers exploit their systems using flaws in their code. The "correct" way to parse that packet would be as follows:

 

void ParseOpcode0()
{
  nameLength = ReadInt();
  if(nameLength < 0 || nameLength > 16) // let's say 16 max characters for name
     // log error and disconnect client
  std::string tmpName;
  tmpName.resize(nameLength);
  ReadArray(&tmpName[0], nameLength);
  for(int x = 0; x < nameLength; ++x)
    // validate character tmpName[x] to ensure it's alpha numeric, _, or any other rules that apply
  // Now validate contents of string, i.e. filter curse words, GM, admin, etc..., make sure first 3 characters are letters, w/e
}

 

Pretty much any message you define, you need to make sure to validate the contents during parsing and be sure to handle all possible problems. This is always easier said than done, but it's only a small part of the challenge of network programming. You also need to take into account Endianness if you are going to be working across different platforms and are going to be sending data back and forth. Otherwise, a value like 0x0000000F on a little-endian machine turns out to be 0x0F000000 on a big-endian machine, which is way off and can wreck havoc on your system (most people run into this problem when interfacing with java servers). Another issue to consider is signed/unsigned data types and how overflow can affect your logic if you do not properly check values.

 

For example, in the code above, since nameLength was an integer, a negative value is possible, so you need to check < 0 and > max. Let's say you had an unsigned value and only checked > max, which would work fine in this case, but if the type ever were to change back to a signed int, then you have a bug in the logic and managing issues like those are a real pain once you enter production. As you program, you have to keep such things in mind and really pay attention to the logic operations you perform on different data types. That was just an example, ideally, your nameLengh would just be an unsigned character to save space and further help prevent overflow issues since 255 is the max size possible.

 

Taking things one step further, packet formats can vary based on flags set in the data. Let's say you have a multi-format packet like this:

word opcode
int id
bool hasName
if hasName
 int nameLength
 char * name
endif
bool hasPosition
if hasPosition
  float x
  float y
  float z
endif

Your processing logic would simply follow the structure and parse out the packet as it was sent. So, without error checking and validation:

id = ReadInt()
hasName = ReadByte()
if(hasName != 0)
{
  nameLength = ReadInt();
  ReadArray(&tmpName[0], nameLength);
}
hasPosition = ReadByte()
if(hasPosition  != 0)
{
  x = ReadFloat();
  y = ReadFloat();
  z = ReadFloat();
}

So, packets can get quite complex. With input driven games like a RTS or FPS, you won't have such complicated packets, but you'd see similar in games that deal with persistent worlds.

 

4) What is object replication, state management and patching (if its not patching in the traditional sense)

 

Object Replication - The ability to create, destroy, serialize, and transmit game objects. RakNet offers the ReplicaManager3 interface for this (I've never used it myself though). NetDog's interface for this can be found here: Objects.

 

State Management - Maintaining a synchronized state of all objects in your game as needed across the network. For example, this is the interface NetDog provides: Object Driven Model Tutorial.

 

Patching - Traditional sense. If you want to have the means to update client files through the game itself (think content downloads, maps, media, etc..) rather than having to do it externally, this is important.

 

Some of these things might seem easy at first glance, but they definitely aren't! Depending on your game type, you will have different needs and requirements to different degrees. In a RTS, you are working with commands being sent over the network, so you don't have much object replication going on. Instead, you have to worry about state management to ensure all simulations are synchronized.

 

Hope that helps some. There is a lot of theory to game network programming and it can get quite complex. There's also a lot of different perspectives and opinions about how things can (or should) be done, so just take this post as one perspective on the matter. There may be some inaccuracies here and there in my post, so make sure to do some more research on anything I've mentioned for other explanations.

 

Good luck!

[gamecreator]

I would also still love to see a simple tutorial showing the basics of networking. I don't need lobbies or patching or chat or anything "higher level" - just reliable transfer of simple data (integers or I guess packets) between two computers.

 

I've looked at RakNet and frankly, it seems more complex than what I need. A lot to learn for "little" need.

[Rekindled Phoenix]

+1 Drew_Benton

You have laid down an excellent foundation for those who need a head start to networking code. Excellent resources btw.

 

I am in the middle of coding an asynchronous multi-threaded networking server, and what Drew has is all true. My process during the creation of a small server seems to be the following:

 

- Decide different 'opcode' commands based on bytes or relevant data

- Create a 'handshake' process exchanging version info and (if necessary) encryption keys

- Put safety checks in place for ALL client data (authenticate occasionally)

 

In my code comments, I have the overall communication setup

   '//Startup Communication//
   'Client: Authenticate Handshake
   'Server: Map and Object Creation
   'Server: Local Vectors

   '//Communication Loop//
   'Client: Keyboard Keys
   'Client: Inventory Requests
   'Client: Chat Messages
   'Server: Local Vector Updates
   'Server: Local Inventory Updates
   'Server: Local Chat Messages
   'Server: Object Updates

 

Here is an article by Tim Sweeney (author of ZZT, my first programming / scripting language) within one of the articles posted above. http://unreal.epicgames.com/Network.htm

 

I'm using .Net, so I have access to API that does a little more than the Leadwerks Networking calls. Plenty can still be done with the given network functions.

[ZioRed]

I'm using .Net, so I have access to API that does a little more than the Leadwerks Networking calls. Plenty can still be done with the given network functions.

For .NET solutions I'm using Lidgren for UDP messaging (even if building a TCP/UDP server class is a relative simple task in .NET with Socket and I used it on an open source project of a game server emulation for DAoC), it's almost complete and very easy to use.