Twitter  YouTube  E-Mail  RSS
The One Man MMO Project
The story of a lone developer's quest to build an online world :: MMO programming, design, and industry commentary
Network Packet Compression
By Robert Basler on 2014-11-27 13:05:30
Homepage: email:one at onemanmmo dot com

Lair Engine uses UDP for messaging. Sometimes it is super-inconvenient to break a message into packet-sized pieces so on top of that, I have what I call JumboPackets. JumboPackets support sending messages larger than a single packet. The cool thing about the way I implemented JumboPackets is that as far as anyone outside the networking system is concerned, they are just like regular packets - only bigger.

The one problem with JumboPackets is that they're subject to rate limiting like any other packet, so if a jumbo packet is actually 3 regular packets, it must be reassembled on the proxy, sent to the client and then reassembled again. So a 3K packet at a 1K/second send rate limit with 2 hops takes 6 seconds to get delivered. This problem gets worse as packets get larger. Recently I started experimenting with much larger datasets (more units, more teams) and discovered that some of the packets sent during the initial connection were getting quite large and noticeably slowing "load time."

A while back I read cbloom's 4 part piece on packet compression in his Oodle networking library (parts 1, 2, 3 and 4.) I was quite impressed by the results he got with his technique.

He settled on LZP compression with a pre-trained static compression table. I suspect (but am not sure) these are different C. Blooms. I found this source implementation of the LZP compressor a little easier to parse.

Looking at the LZP code, it seemed like more trouble to implement than I wanted to spend on an experiment. Fortunately the first cbloom had done some testing on compression rates of LZP versus ZLIB and his primary problem with ZLIB was its memory use. I'm not at the point where memory use is my biggest concern and I already had ZLIB implemented, so for now I'm using ZLIB instead on level 1 for fastest compression and smallest memory use.

Some day maybe I'll implement cbloom's system for all packets, but even just compressing JumboPackets the results with ZLIB are good. Most packets are shrunk 60% or more.

New Comment

Cookie Warning

We were unable to retrieve our cookie from your web browser. If pressing F5 once to reload this page does not get rid of this message, please read this to learn more.

You will not be able to post until you resolve this problem.

Comment (You can use HTML, but please double-check web link URLs and HTML tags!)
Your Name
Homepage (optional, don't include http://)
Email (optional, but automatically spam protected so please do)
Type boy. (What's this?)

  Admin Log In

[Home] [Blog] [Video] [Shop] [Press Kit] [About]
Terms Of Use & Privacy Policy