Caching steam depots on your local server with nginx and dsniff

While I usually don’t advertise non libre software for obvious reasons (that’s a stupid way to think about computing), I admit, though, that the Steam platform goes toward what I’d like to see since many years. Proprietary software platform indeed – but the business is not made out of selling overly expensive DVD-Rom once in a while but cheap soft (downloadable) copies of games (often) maintained over years. They also seem about to base a future gaming console on some sort of GNU/Linux flavor, that’s not philanthropy, that’s just the only clever way to do a cool gaming based business without getting totally dependant on another software supplier that also brand his own gaming console. Latest South Park was about the fight beetween latest Xbox and Playstation. This issue only exists when you decide to make console non compatible with usual workstation, a shortcut with so many shortcomings. Making a GNU/Linux based console, because it is good business, is obviously going in the right direction.

So I’ll allow myself a little reminder here on how not to waste your bandwidth on a local network where you have several computers having copies of the same steam game. It’s merely a simplified version of the well thought Caching Steam Downloads @ LAN’s article. Obviously, to do this, you need to have your own home server. For instance, it should work out of the box with a setup like this (this is the setup mentioned before from now on in this article).

A) HTTP setup

We first create a directory to store steam depot. It will be served with http so you need to  create something like (working with the setup mentioned before):

mkdir /srv/www/depot
chown www-data:www-data /srv/www/depot

Next, you want to setup nginx, to be able to serve as a steam content provider. Everything is based on http -no proprietary non-standard crap- so it can only go smoothly.

If you have the setup mentioned before, then  /etc/nginx/sites-available/default contains a server { } statement for general intranet. Add a new file called /etc/nginx/sites-available/steam with the following (watch out for the listen and allow statements, change it depending on your server intranet IP!):

# steam spoof/proxy
server  {
  # you want this line to be set to your server intranet IP
  listen 10.0.0.1;
  listen 127.0.0.1;
  server_name *.steampowered.com;

  access_log /var/log/nginx/steam.access.log;
  error_log /var/log/nginx/steam.error.log;

  root /srv/www/;
  resolver 127.0.0.1;
  ## use this resolver instead if your local interface got domain names spoofed 
  ##resolver 8.8.8.8;

  # restrict to local wired network
  allow 10.0.0.0/24;https://github.com/yeupou/stalag13/blob/master/etc/cron.daily/cache-steam
  allow 127.0.0.1;
  deny all;
  location /depot/ {
    try_files $uri @mirror;
  }

  location / {
    proxy_next_upstream error timeout http_404;
    proxy_pass http://$host$request_uri;
    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded_For $proxy_add_x_forwarded_for;
    add_header X-Mirror-Upstream-Status $upstream_status;
    add_header X-Mirror-Upstream-Response-Time $upstream_response_time;
    add_header X-Mirror-Status $upstream_cache_status;
  }

  location @mirror {
    access_log /var/log/nginx/steam.remote.log;
    proxy_store on;
    proxy_store_access user:rw group:rw all:r;
    proxy_next_upstream error timeout http_404;
    proxy_pass http://$host$request_uri;
    proxy_redirect off;
    proxy_set_header Host $host;
    proxy_set_header X-Real-IP $remote_addr;
    proxy_set_header X-Forwarded_For $proxy_add_x_forwarded_for;
    add_header X-Mirror-Upstream-Status $upstream_status;
    add_header X-Mirror-Upstream-Response-Time $upstream_response_time;
    add_header X-Mirror-Status $upstream_cache_status;
   }
}

Make it live:

cd /etc/nginx/sites-enabled && ln -s ../sites-available/steam .
invoke-rc.d nginx restart

Now nginx is able to fetch and serve steam depot files.

B) DNS setup

Now, you need your server to actually handle requests to steam content server, spoofing these servers IPs. It could be done by messing with the DNS cache server already up on the setup mentioned before but I actually find much more convenient to use dnsspoof from dsniff package with a two-line configuration than wasting time creating say bind9 unnecessarily complex db files.

So we first instead dnsspoof:

apt-get install dsniff

Here come’s the two line configuration, set in /etc/dnsspoof.conf. Obviously, here too you have to set the IP to be your server’s intranet one.

10.0.0.1     *.cs.steampowered.com
10.0.0.1     content*.steampowered.com

Then you want an init.d script. You can create an ugly /etc/init.d/dnspoof with the following (obviously, you want you ethernet ethX device to be properly set!):

/#! /bin/sh
### BEGIN INIT INFO
# Provides:          dnsspoof
# Required-Start:    bind9
# Required-Stop:     
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: Start dnsspoof over bind9
# Description:       Start dnsspoof over bind9
### END INIT INFO

# shitty version for testing purpose
/usr/sbin/dnsspoof -i eth1 -f /etc/dnsspoof.conf 2> /dev/null > /dev/null &

# EOF

Or, 5th December UPDATE, you can download my init script doing as follows:

cd /etc/init.d && wget https://github.com/yeupou/stalag13/raw/master/etc/init.d/dnsspoof && chmod +x dnsspoof && update-rc.d dnsspoof defaults
cd /etc/default && wget https://github.com/yeupou/stalag13/raw/master/etc/default/dnsspoof

(dont forget to edit /etc/default/dnsspoof to suit your setup, especially regarding the ethernet ethX device to be used)

Once ready, just start the spoofer:

chmod 755 /etc/init.d/dnsspoof
invoke-rc.d dnsspoof start

Now you can restart steam on your clients computers. It should work properly. You can check whether new directories appear in /srv/www/depot and monitor /var/log/nginx/steam* logs.

I’ll soon add a small script to get more meaningful info about the depots available on your server, so you can know which are what in a jiffy and remove the no longer useful willy-nilly.

December 10th update: soon is now. Get the script doing the following, if you run the setup mentioned before, it should work out of the box:

cd /etc/cron.daily && wget https://github.com/yeupou/stalag13/raw/master/etc/cron.daily/cache-steam && chmod +x cache-steam
./cache-steam

This should update  a daily depots.html file in your /srv/www/depot/.

Advertisements

Counter-Strike/Day Of Defeat userconfig.cfg and the neverending quest for a Libre Software business model regarding games

Baby you can come along too, but I'm getting drunk without or without you! - Gettin' Drunk by the Beat Farmers

I played to most of the famous first-person shooter (FPS) of the 1990’s and some of the 2000’s: Wolfenstein 3D, Doom, Doom II, Duke 3D, Quake, Half-Life, Medal of Honor, even the less famous like Redneck Rampage (shipped with a wonderful country/rock-a-billy/psychobilly soundtrack; still not kidding, I would even consider going to Reverend Horton Heat gig that will take place in France in a few days if I was rich, which I’m not) etc… But I played regularly for years only to a few of them.

Duke 3D was unmistakenly fun and definitely made the best out of numerous hours of LAN games. Counter-Strike’s (CS) gameplay, however, remains unique. Obviously, the rendering engine was technically modern enough (Duke 3D is fun, but you do not need to aim target in 3D, you can destroy a target up in the air while aiming the boardwalk ; in Doom II, you cannot even jump). But that’s not just it, no, the game really leads to team play. The two teams, counter-terrorists or terrorists, both have an obvious objective (blowing up a bomb, freeing hostages, etc) and each teammate that goes down is out of the game for the round. So unlike in Medal of Honor, Quake Arena, etc (where one player does not equal one another), any player matters. To some players, this a drawback, they rather have unlimited lives, moving fast, dying fast, respawning fast, moving fast, dying still fast. To me, it is part of what makes this game aside from the others FPS. Based of the same rendering engine (the one of Half-Life), Day of Defeat (DoD) and Team Fortress (TF) both provides some kind of team play also well thought but way more conventional (capture the flag, mainly), with respawn. Day of Defeat takes place during World War II while Team Fortress is kind of cartonish/fun, somehow reminding me of Duke3D.

Counter-Strike with Source engine, still not outdated


Half-Life² was eventually published with a new rendering engine called Source. And it is a good thing they also released CS, DoD and TF renewed with this new engine, so these good games kept up with the trend. Despite the fact that the map oilrig (a mode where you had to protect a somekind of VIP) disappeared during the transition to Source, CS gameplay was unaffected. And I continued playing from time to time to CS. I still do.

Today, I found out that I no longer had my userconfig.cfg. In CS, you need to buy weapons at the begin of each round. And, clearly, when a round begins, you have no time to fool around, wasting precious seconds by looking at each possible weapon. You have to take the best you can get in the fastest way possible, which meaning having shorcuts. Such shortcuts are not provided, so you have to write them in a userconfig.cfg file. As, unfortunately, we are not speaking of Libre Software here, everything is really poorly documented, you have to google around. So here comes my cstrike userconfig.cfg:

// ---------------------------------------------------------------
// shopping: commands

// gun
alias buy_gun "buy p228; buy secammo; buy vesthelm;"
// shotgun
alias buy_shotgun "buy xm1014; buy primammo; buy vesthelm;"
// cheapeast decent
alias buy_cheapest "buy mp5; buy primammo; buy vesthelm;"
// assault rifles
alias buy_cheap "buy galil; buy famas; buy primammo; buy vesthelm;"
alias buy_expensive "buy ak47; buy m4a1; buy primammo; buy vesthelm;"
alias buy_scope "buy sg552; buy aug; buy primammo; buy vesthelm;"
// equipment
alias buy_protection "buy vesthelm; buy vest"
alias buy_fb "buy flashbang; buy flashbang"
alias buy_he "buy hegrenade"
alias buy_smoke "buy smokegrenade"
alias buy_defuser "buy defuser"
alias buy_ammo "buy primammo; buy primammo; buy secammo"
alias buy_equipment1 "buy_protection; buy_defuser ; buy_ammo"
alias buy_equipment2 "buy_protection; buy_fb; buy_smoke; buy_he"

// shopping: binds

bind "F1" "buy_equipment1"
bind "F2" "buy_equipment2"
bind "F3" "buy_gun"
bind "F4" "buy_shotgun"
bind "F5" "buy_cheapest"
bind "F6" "buy_cheap"
bind "F7" "buy_expensive"
bind "F8" "buy_scope"

// ---------------------------------------------------------------
// communication: commands
// unfortunately, we have no variable to say exactly where we are
// so it is meaningless to provide much details

alias waitalot "wait;wait;wait;wait;wait;wait;wait;wait;wait;wait;wait;"
alias w "waitalot;waitalot;waitalot;waitalot;waitalot;waitalot;"
alias msg_enemies_none "radio3; w; slot4"
alias msg_enemies_few "radio3; w; slot2"
alias msg_in_a "say_team (A !!!)"
alias msg_in_b "say_team (B !!!)"
alias msg_bomb_here "radio3; w; slot3; w; say_team The bomb is here!"
alias msg_yes "radio3; w; slot1"
alias msg_no "radio3; w; slot8"
alias msg_follow_me "radio1; w; slot5"

// communication: binds

bind "KP_END" "msg_enemies_none"
bind "KP_DOWNARROW" "msg_enemies_few"
bind "KP_PGDN" "msg_enemies_plenty"
bind "KP_LEFTARROW" "msg_in_a"
bind "KP_5" "msg_in_b"
bind "KP_RIGHTARROW" "msg_bomb_here"
bind "KP_HOME" "msg_yes"
bind "KP_UPARROW" "msg_no"
bind "KP_PGUP" "msg_follow_me"

// EOF

In Day of Defeat, the userconfig.cfg matters less, as you do not have to buy weapons. However, the communication shortcuts can be of use, especially as in DoD:S, unlike in CS:S, you have access to the variable providing your location on the map. Here comes my dod usercfg.cfg making use of the location %l variable:

// ---------------------------------------------------------------
// communication: commands
// at the contrary of CS:S, we have the location
// variable (%l) that enables us to
// provide detailed info

alias msg_enemies_none "voice_areaclear; say_team %l clear"
alias msg_enemies_ahead "voice_enemyahead; say_team Enemy ahead %l"
alias msg_enemies_behind "voice_enemybehind; say_team Enemy behind us at %l"
alias msg_enemies_left "voice_fireleft; say_team To my left at %l"
alias msg_enemies_right "voice_fireright; say_team To my right at %l"
alias msg_enemies_machinegun "voice_mgahead; say_team MG at %l"
alias msg_enemies_sniper "voice_sniper; say_team Sniper at %l"
alias msg_enemies_grenade "voice_grenade"
// do not use voice_backup here because it may provide to much insight to the other team
alias msg_needhelp "say_team I NEED BACKUP AT %l"
alias msg_comeon "voice_gogogo"

// communication: binds

bind "KP_END" "msg_enemies_left"
bind "KP_DOWNARROW" "msg_enemies_ahead"
bind "KP_PGDN" "msg_enemies_right"
bind "KP_DEL" "msg_enemies_behind"
bind "KP_LEFTARROW" "msg_enemies_machinegun"
bind "KP_5" "msg_enemies_sniper"
bind "KP_RIGHTARROW" "msg_enemies_grenade"
bind "KP_HOME" "msg_needhelp"
bind "KP_UPARROW" "msg_comeon"
bind "KP_PGUP" "msg_enemies_none"

// EOF

And this dichotomy between CS:S and DoD:S, both based on the same rendering engine, definitely illustrates how frustrating proprietary software can be. If %l exists in DoD:S, it cannot be complicated to implement in CS:S. It is not really an issue of time, if anybody was able to read the code, edit it and redistribute the modifications made, there would have been an user willing to spend this time. I know I would.
But it is not and so we just have to rely on CS:S publisher that does not show much interest in developing further CS:S.

eduke3d and STALKER COP: lights, shadows, objects, textures, no room for comparison

Clearly, there is no recent and great piece of FPS that is Libre Software that I know of. We just get old rendering engine GPLed: Ken Silverman’s Build (Duke 3D/Shadow Warrior/Redneck Rampage/etc engine), Id Software’s id Tech 3 (Quake III, MOHAA, Call of Duty/etc). Even if it is nice that this code is freed at last, I have the sad feeling that they would not be freed if their authors felt their code could be someday improved enough to compete with their own new rendering engine. So no business can be based on these rendering engines, and we’re stuck with Libre Software looking like cheap obsolete products. It may sound harsh, don’t get me wrong: I enjoy the work being done on eduke32, that’s nice, and I’m not criticizing the efforts made. Still, it cannot compete with what is being published as proprietary sofware. Even games with somehow outdated rendering engines like STALKER Call of Pripyat are way ahead.
When Half-Life² was published, there was hope: their business model seemed to implies releasing frequently data (episodes – levels, including artwork). They could have freed their engine. But they didn’t and I guess they just failed to see any profit in doing so – well, I could say that now CS:S would handle %l 🙂

I’m wondering if we’ll ever see brand new games published as Libre Software. The only people that have the cash flow and the motives to do so could be ATI or NDVIDIA, the hardware producers. But they may has well just continue to work as they currently do.

(PS: and I’m not even going as far as expecting these to run under a GNU-based operating system)

Transport Tycoon nowadays

Do you remember back in the days of 1996/1997, fooling around with a Pentium 100 MMX and 32 MB RAM?

A small depot on the way...


Among Cannon Fodder, Dune 2000, Duke 3D, there was Transport Tycoon (Deluxe), « a business simulation game, presented in an isometric view in 2D […], in which the player is in control of a transport company, and can compete against rival companies to make as much profit as possible by transporting passengers and various goods by road, rail, sea or by air » as described by Wikipedia.

Nice gameplay, nice graphics (considering it was no CGA or EGA) with buildings inspired from Glasgow, it even featured network game ability.

and a cheap train with one purpose...


At that time, networking-wise, I knew nothing but null-modem, which was a real nightmare of instability in conjonction with MS Windows 95. So I enjoyed 1vs1/human vs human once in a while, when it was working. Not to mention that’s the kind of game where it is way easier to fight the IA players, more or less allied -or at least not interfering- with the other human player. Obviously when you spend nearly 30 minutes to design a nice railroad junction, you will easily loose temper when the guy sitting 3 meters away is messing it up, making it unprofitable and whatever. It is definitely easier to mess with the IA players that are dumb enough so you can destroy at will their bus/trucks, with a tiny train depot placed near their truck depots/destination, a cheap train. The only issue I ever had with this game are planes: they are way too profitable (at mid-game) with almost no effort, so there is no point in continuing spending hours to lay tracks… but isn’t it the purpose of the game, to design complicated tracks design? What’s the fun in putting 2 bigs squares inside busy cities?

After TTD, several games more or less similar were published: Railroad Tycoon (focused on the trains – maybe sharing my opinion about the planes?), Railroad Tycoon 2, Railroad Tycoon 3, Railroads, Chris Sawyer’s Locomotion (this one being made by Chris Sawyer, author of TTD).

I’ve tried them all. Chris Sawyer’s Locomotion is a nicely improved version of TTD: better graphics, slight gameplay improvements. It would have been great if published in 1999/2000. But it was published late in 2004 and way too much user interface progress have been made since then – laying tracks still in the old TTD way was no longer an option.

The Railroad Tycoon (RT) serie, in regard of user interface and aesthetics, was clearly superior. Nice scenarii, great country music (this is not a joke), clearly you couldn’t wait to enjoy network play… But you should have. It was buggy, subject to plenty of loss of synchronization – meaning the networked computers lost touch of what each other was doing. And, that was a hit when I found out, it handled in the most stupid way these lost of sync: it was simply disregarding the issue, and, unaware, you were continuing to play, laying tracks, whatever, until you found out that you were no longer playing exactly the same game that the other player. You had layed tracks from the Seattle to Minneapolis, it was working good – but it existed only in your instance, it would not show up on the other player computer. And, at some point, you had a segfault from one or the other instance, probably because of a conflict between the two concurrent ongoing games. Depending on which computer you grabbed the savegame from, to restart where it crashed, you could see the gap between the two concurrent games… This bug was present in the whole RT serie. The game support was a joke, a few patches were published (few days after RT3 release – which says a lot about the extent of the test cycle that the publisher probably made before the release) but none really fixed it. And as it is proprietary software, there is no legal room for improvement.

So far, there is no TTD-like game that fit to modern user interface standards while allowing decent network play.

So, here comes OpenTTD. TTD was written 99% in assembler, some lads decided to reverse-engineer it and, tada!, they published the result under the GNU GPL 2.

Internet/LAN play in OpenTTD


Taken from their about page, significant enhancements from the original game include for example: bigger maps (up to 64 times in size), stable multiplayer mode for up to 255 players in 15 companies or as spectators, dedicated server mode and an in-game console for administration, new pathfinding algorithms that makes vehicles go where you want them to, autorail/-road build tool, improved terraforming canals, shiplifts, aqueduct larger, non-uniform stations and the ability to join them together, mammoth and multi-headed trains clone, autoreplace and autoupdate vehicles, possibility to build on slopes and coasts, advanced/conditional orders, share and copy orders, longer and higher bridges including new ones, full flexible tracks/roads under bridge , reworked airport system with many more airports/heliports (i.e. international and metropolitan), presignals, semaphores, path based signalling, support for TTDPatch newgrf features, drive-through road stops for articulated road vehicles and trams, multiple trees on one tile, bribe the town authority, […] convert rail tool (to e-rail, monorail, and maglev), drag&drop support for almost all tools (demolition, road/rail building/removing, stations, scenario editor…), sorting of most lists based on various criteria (vehicle, station, town, industries, etc.), autoscroll when the mouse is near the edge of the screen/window, cost estimation with the ‘shift’ key, etc.

Obtaining content with BaNaNaS inside OpenTTD

In the interface, you can even fetch online new features, scenarii or whatever with BaNaNaS, a content service. And that is the point of this article 🙂
You can find there two scenarii I’ve made for OpenTTD: one is called France and is a map of France, the other one is called Jungle Urbaine (Paris) and is a map of Paris. In both these maps, industries are placed randomly.

France OpenTTD scenario as available on BaNaNaS


Jungle Urbaine (Paris) OpenTTD scenario as available on BaNaNaS