[img]http://dl.dropbox.com/u/99606/steamre.png[/img]
Hello all,
I decided to make a thread to inform everyone about some recent developments that have happened with the SteamRE project.
For those of you who haven't heard of SteamRE (and I assume that is a large majority), it is quite simply an ongoing effort to understand the physical networking behind Steam itself.
The team behind it includes [url=http://www.facepunch.com/members/197912-Chrisaster]Chrisaster[/url], [url=http://www.facepunch.com/members/136199-deloc]stgn[/url], [url=http://www.facepunch.com/members/28539-ComWalk]ComWalk[/url], and myself.
Some of you may remember a [url=http://www.facepunch.com/threads/824685-Open-Steamworks?p=23671605&viewfull=1#post23671605]few posts I've made in the Open Steamworks[/url] thread about the project.
Needless to say, the project got very far initially and then, school started and all progress pretty much stopped. The SteamRE project didn't move anywhere until a few days ago when the "Midwest Blizzard of 2011" gave me a few days free time to get some work done.
Today I want to unveil some fruits of all our efforts that the team has put in. We've tinkered with the protocol enough that we have a pretty clear understanding of how the protocol itself works, and we're able to achieve full log on, and friend messaging capabilities.
We've developed a heavily WIP C# library called SteamKit2 that implements the networking, and allows developers the control to make their own clients. We've also developed our own POC client to test out code as we write it.
The code is freely available at this Bitbucket proejct: [url]https://bitbucket.org/VoiDeD/steamre[/url].
Keep in mind that this project is a WIP, and if you're going to use SteamKit2 in your projects, there will be breaking changes in the future. The library is nowhere near stable at this point, but we're going to continue working on it.
[release]Here's a video that shows the proof of concept client in action: [url]http://dl.dropbox.com/u/99606/osw_stuff/2011-02-03_1720.swf[/url]
A few things to note here, this is running on Ubuntu. This is an entirely stand alone client. I've [url=http://www.facepunch.com/threads/824685-Open-Steamworks?p=22388781&viewfull=1#post22388781]said that before[/url], but this time it's [i]completely[/i] stand alone and doesn't require any libraries or dependencies on steam.
Potentially you can use this client on any operating system that supports Mono.[/release]
[editline]3rd February 2011[/editline]
I forgot to mention some contact details.
The primary place we use for discussing development is the opensteamworks IRC channel.
You can find it on [url=irc://irc.gamesurge.net/opensteamworks]irc.gamesurge.org @ #opensteamworks[/url].
Holy shit, finally a way for me to easily talk with steam friends without having to start up steam itself.
Now for an app :v:
But useful as shit anyway.
I would like to write an Android chat client, got any documentation?
[editline]3rd February 2011[/editline]
Btw this is awesome
[QUOTE=foszor;27842525]I would like to write an Android chat client, got any documentation?
[editline]3rd February 2011[/editline]
Btw this is awesome[/QUOTE]
The documentation, at this point, is entirely the code we have available. We used to have a wiki run by stgn, but it's gone now. Your best bet is following the POC client code.
Also, I'm not entirely sure how you'd get it running on Android, since the current library requires Mono.
[QUOTE=VoiDeD;27842753]The documentation, at this point, is entirely the code we have available. We used to have a wiki run by stgn, but it's gone now. Your best bet is following the POC client code.
Also, I'm not entirely sure how you'd get it running on Android, since the current library requires Mono.[/QUOTE]
I guess thats the reason he requested documentation, so he could translate your findings in to Java.
Also would've thought reverse engineering was against TOS.
Isn't this thread more suitable for a certain Russian counter-strike source site?
Ergh, that receive code is making me go crazy.
[QUOTE=high;27843509]Ergh, that receive code is making me go crazy.[/QUOTE]
You should have seen the UDP code in the old SteamKit. What a disaster that was...
For some reference, Valve's Steam3 implementation in Steam makes use of UDP by default, but they practically recreated TCP's reliability on top of it.
[url]http://tracker.limetech.org/projects/steamre/repository/revisions/113/entry/SteamKitX/Network/UDP/UdpConnection.cs[/url]
I'm not much of a networking guru, but it seems to me that you may as well have used TCP at that point.
Hopefully I can put this on a USB to talk to my friends on steam when I'm away from my home computer. :-)
[QUOTE=VoiDeD;27843677]You should have seen the UDP code in the old SteamKit. What a disaster that was...[/QUOTE]
old steamkit was a disaster in general. it wasn't even a library and passwords in steam3 were hardcoded at one point.
[QUOTE=VoiDeD;27842753]since the current library requires Mono.[/QUOTE]
Monodroid
[QUOTE=SteveUK;27844126]Monodroid[/QUOTE]
embedding mono yourself would probably be more worth it.
[QUOTE=VoiDeD;27843677]You should have seen the UDP code in the old SteamKit. What a disaster that was...
For some reference, Valve's Steam3 implementation in Steam makes use of UDP by default, but they practically recreated TCP's reliability on top of it.
[url]http://tracker.limetech.org/projects/steamre/repository/revisions/113/entry/SteamKitX/Network/UDP/UdpConnection.cs[/url]
I'm not much of a networking guru, but it seems to me that you may as well have used TCP at that point.[/QUOTE]
Rewriting "NetLoop" because it is going to bug me until I do.
Also why do you have DataStream? Why not just use BinaryReader? Only reason I can see why is because BinaryReader doesn't have a Read<Type>() but you can always add it as an extension or create a class that inherits from it and adds it.
[QUOTE=high;27844355]Also why do you have DataStream? Why not just use BinaryReader? Only reason I can see why is because BinaryReader doesn't have a Read<Type>() but you can always add it as an extension or create a class that inherits from it and adds it.[/QUOTE]
Gives me convenience of not having to create a MemoryStream and BinaryReader for every buffer I need to read out of, and handles swapping endianness.
Any chance that an iPhone steam client could be made with this?
[QUOTE=cheesywlz;27844587]Any chance that an iPhone steam client could be made with this?[/QUOTE]
monotouch, embed mono yourself, or port it to obj-c (eugh).
[QUOTE=foszor;27842525]I would like to write an Android chat client, got any documentation?
[editline]3rd February 2011[/editline]
Btw this is awesome[/QUOTE]
God dammit I was going to do that.
[QUOTE=geel9;27844749]God dammit I was going to do that.[/QUOTE]
hahahaha good luck with that.
[QUOTE=VoiDeD;27844486]Gives me convenience of not having to create a MemoryStream and BinaryReader for every buffer I need to read out of, and handles swapping endianness.[/QUOTE]
Can always create a constructor for BinaryReader that takes a byte[] :P. Although for endianness you would end up needing to create a class anyways.
Anyways, I rewrote NetLoop. Personally I would create something so that it would be a bit cleaner. Like a class that handles receiving X bytes instead of the 2 loops. But I didn't want to change too much. This is more just for showing. Anyways I did test it and it does work(Worked the first time too :D).
[csharp]void NetLoop()
{
while (true)
{
lock (ConnLock)
{
if (!bConnected)
return;
}
MemoryStream buf;
int read;
byte[] tmp = new byte[1024];
buf = new MemoryStream(8);
read = 0;
while (buf.Position < buf.Length && (read = sock.Receive(tmp, Math.Min(tmp.Length, (int)(buf.Length - buf.Position)), SocketFlags.None)) != 0)
buf.Write(tmp, 0, read);
if (read == 0)
{
DebugLog.WriteLine("TcpConnection", "Recv'd connection closed!");
return;
}
DataStream ds = new DataStream(buf.ToArray());
uint packetLen = ds.ReadUInt32();
uint packetMagic = ds.ReadUInt32();
if (packetMagic != TcpConnection.MAGIC)
{
DebugLog.WriteLine("TcpConnection", "RecvCompleted got a packet with invalid magic!");
return;
}
buf = new MemoryStream(packetLen);
read = 0;
while (buf.Position < buf.Length && (read = sock.Receive(tmp, Math.Min(tmp.Length, (int)(buf.Length - buf.Position)), SocketFlags.None)) != 0)
buf.Write(tmp, 0, read);
if (read == 0)
{
DebugLog.WriteLine("TcpConnection", "Recv'd connection closed!");
return;
}
byte[] bufa = buf.ToArray();
if (NetFilter != null)
{
bufa = NetFilter.ProcessIncoming(bufa);
}
OnNetMsgReceived(new NetMsgEventArgs(bufa));
}
}[/csharp]
[QUOTE=ghettochip;27844777]hahahaha good luck with that.[/QUOTE]
What does this mean, exactly?
[QUOTE=VoiDeD;27842753]The documentation, at this point, is entirely the code we have available. We used to have a wiki run by stgn, but it's gone now. Your best bet is following the POC client code.
Also, I'm not entirely sure how you'd get it running on Android, since the current library requires Mono.[/QUOTE]
I guess I was more interested in writing it from the ground up, without the use of your library. Not entirely sure it's possible, was just curious to try.
[QUOTE=foszor;27846863]I guess I was more interested in writing it from the ground up, without the use of your library. Not entirely sure it's possible, was just curious to try.[/QUOTE]
We're writing up some preliminary documentation that should explain the networking more than the code will.
[QUOTE=VoiDeD;27846960]We're writing up some preliminary documentation that should explain the networking more than the code will.[/QUOTE]
Awesome
[QUOTE=geel9;27845064]What does this mean, exactly?[/QUOTE]
[img]http://ahb.me/1HMc[/img]
[QUOTE=Matthew0505;27847724]You'd still need to get a Steam account with the game bought to play on VAC-enabled servers, so not really[/QUOTE]
Couldn't someone just take this work, put in some effort, and make their own steam servers(give me boxes, I have no idea what I'm talking about here)
[QUOTE=mmavipc;27848165]Couldn't someone just take this work, put in some effort, and make their own steam servers(give me boxes, I have no idea what I'm talking about here)[/QUOTE]
That idea in itself is interesting, but honestly holds no merit. I don't condone or like the idea of Steam piracy, but in practice it isn't very hard to patch a few checks in Valve's client.
[QUOTE=VoiDeD;27846960]We're writing up some preliminary documentation that should explain the networking more than the code will.[/QUOTE]
Alright, we've gone through and explained the gist of the inner workings and some other things.
You can view the docs -snip. There is also an easier to view version -snip-
Keep in mind that you'll have to reference the code in order to understand some parts of this. Also, if anyone has any questions regarding anything please go ahead and mention them so we can clarify.
[b]Edit[/b]: The docs have moved to [url]http://tracker.limetech.org/projects/steamre/wiki/[/url]
This is pretty impressive - as others have said in this thread, I'll find this handy for chatting to Steam friends when not on Steam.
Thank you for your work guys, hope you have been finding it interesting.
[QUOTE=VoiDeD;27843677]You should have seen the UDP code in the old SteamKit. What a disaster that was...
For some reference, Valve's Steam3 implementation in Steam makes use of UDP by default, but they practically recreated TCP's reliability on top of it.
[url]http://tracker.limetech.org/projects/steamre/repository/revisions/113/entry/SteamKitX/Network/UDP/UdpConnection.cs[/url]
I'm not much of a networking guru, but it seems to me that you may as well have used TCP at that point.[/QUOTE]
It's actually very common to implement TCP-like stream protocols over UDP for games. When it comes down to it, TCP isn't very well suited for most modern games at all. The biggest problem is that TCP doesn't give you fine grained control over data priorities, since all data is assumed top priority. TCP will make sure that all data reaches its destination, no matter how much time passes, and will also make sure that data is received in the order it was sent. In data-intensive real-time games, like FPS games etc, neither of these guarantees are very important, hence UDP is used as the base for a custom protocol, often coupled with a separate TCP connection for stuff like chat and transfer of large data chunks (like game resources).
Sorry, you need to Log In to post a reply to this thread.
