[t]http://i45.tinypic.com/34zis69.png[/t]
[editline]17th January 2013[/editline]
[t]http://i48.tinypic.com/2rc5081.png[/t]
and going the other way
So, after I got bored of writing a UI library from scratch in Java for a game I was going to work on, I randomly got the idea of attempting to build a simple AI.
The concept of machine learning has always been fascinating to me, and has in recent months really caught my eye - I intend to take a course on it as an elective when I am next able to. So, I decided to build one.
The AI itself is named SAL ([b]S[/b]imple [b]A[/b]rtificial Intelligence that [b]L[/b]earns), and as its name implies, its learning method is very simple.
Basically, SAL operates on two separate principles: a decision engine, and a learning engine. The whole thing relies on the fact that I create some system (testing with a really simple game I threw together) that supports SAL in the way of a decision-tree and an AI-wrapper. The decision tree, as the name implies, is a tree that handles all of the possible decisions that could be made, and the wrapper translates those decisions into actual functions, so that SAL can play in the place of a human. Let me explain the decision tree first, and then the learning system.
Obviously, the [b]decision tree[/b] grows ridiculously huge as the system being tested grows in complexity, as does the wrapper (or decision-translator, as I refer to it).
With that being said, there are two elements in the decision tree: Leaves and nodes. A [b]leaf[/b] is the very bottom-most end of a tree branch; there are no farther branchings beyond a leaf. A leaf is also known as an "executive decision." A [b]node[/b] is simply a sub-array, which contains other leaves and nodes. A node is known as a "branching decision." Nodes and leaves are collectively known as [b]choices[/b].
Every node and leaf in the decision tree has a weight. SAL's decision engine, for any given level, simply sums up the weights of everything on that level, and then picks a random floating number (up to 4 digits of precision) between 0 and that max weight. It will then iterate through all the choices on that level, summing their weights, until it reaches a point where the summed weight exceeds the chosen weight. Whatever choice causes this excess is chosen.
If the chosen element is a node, then the decision engine recurses on that level, and the engine continues to recurse until a leaf is chosen - at which point, the leaf is tossed through the translation engine, and the function related to that leaf is called.
[b]At the moment[/b], I have the decision engine completed. The learning system has its foundation in place, and I just need to actually put it all together. The [b]learning system[/b] is equally simple (again, hence the name), and operates in the following manner:
When a new instance of SAL is created, it is birthed completely fresh, with no bias at all; every single decision is equally weighed. When a learning session begins (for example, starting a game against another player), SAL chooses a random decision and modifies that decision's weight. The amount of change is dependent on SAL's [b]confidence[/b], which I will explain shortly.
When SAL makes a change to a decision's weight, the decision being modified, and the modification amount, is recorded and stored in a "temporary buffer." SAL will then proceed through the session with the augmented weights. When the session concludes, SAL is given an evaluation. This evaluation is centered around a "desired-outcome" factor: A DOF of 1.0 reflects 100% desirable outcome (a flawless game, for example), whereas a DOF of -1.0 reflects a 0% desirable outcome (absolute defeat, for example). 0.0 is considered "perfectly average."
The DOF is added to a "desirability sum", and the "total iterations" is incremented by one. A "desirable-outcome ratio" is calculated, with a simple ratio of "desirability sum" / "total iterations". From that, SAL's new confidence is determined.
Then, all the augmentations made at the beginning of the session are undone, those changed factors are multiplied by that session's DOF, and then the modified values are added permenantly into SAL's decision tree. When the next session begins, the whole process takes place again.
The purpose for the [b]confidence[/b] system is to moderate the severity of SAL's changes. When SAL first starts, he has no confidence in his changes, because he has no previous experience, so his changes fluctuate wildly. But as SAL does better as a result of his changes, he grows more confident in his current weights. As a result of his confidence, he makes smaller changes in future sessions. As SAL's confidence in his ability approaches 100%, the changes he makes falls into fractions of percentages - effectively, SAL follows the adage of "don't fix what isn't broken."
But the confidence swings both ways. If SAL's changes result in his constantly performing poorly, his confidence in his ability will drop - possibly into the negatives - and will result in him wildly changing his probabilities. He effectively grows desperate, and attempts to try anything to dig himself out of the hole. Hopefully, he will eventually randomly sling himself into something that improves his performance, building his confidence up and putting him back on track.
The game that I am building for SAL to learn on is a really simple strategy board-game, with three different ways to win: one that rewards attacking the enemy base with units, one that rewards defending his own base with units, and one that rewards focusing on resources over units. With these three distinct ways to win, it will be interesting to see what type of strategy SAL develops.
Currently, SAL's learning system only supports the experimenting of a single decision at a time. However, once I have the base learning system down, I intend to complicate the learning engine by allowing him to make changes to multiple decisions at once, and during evaluation, be able to weigh the impact of each change and make permanent adjustments accordingly. How I would perform such impact-weighing I am not sure on right now, though.
I hope to one day be able to pit two iterations of SAL against each other, entirely untrained, and watch them learn off each other and see what kind of personalities they develop. Other far-flung ideas are to provide a sort of crowdsurf training, where human players can play against one of a series of different SAL AIs over the Internet, teaching the AI as they do so.
I know this is a long post, but I felt as though I should explain in as much detail what SAL is and what his plans are.
Here is a screenshot of debug of his decision-engine. The values with "= 1" are leaves, with 1 being the weight (this is without any learning, so everything is weighed evenly). The Special leaf of _WEIGHT is the weight of the associated node.
[img]https://dl.dropbox.com/u/8416055/SAL_WIP_003.jpg[/img]
And, yes, I am building SAL in Lua / Love2D because, frankly, I find Lua to be a great language to proof-of-concept things in. It's fast and easy to work with, without having to worry about all the details other programming languages force on you, like specific datatypes and such. I may convert SAL to another, more mainstream language, once I get him working.
So yeah, this is what I have been spending my time doing instead of homework.
SAL-9000?
I had an idea for a Bomberman type game. But before I do that, I need to know how to do networking.
So, networking and XNA devs, what do you use?
asynchronous or synchronous?
TCP or UDP?
System.Net.Sockets.Socket?
Asynchronous if you don't want your game to freeze every 50ms.
TCP if you're lazy, UDP if you want speed.
For simplicity I would say just use a standard TCP socket which should suffice, but ideally UDP would be better if you have experience with it, or a mix of TCP/UDP.
Personally I use RakNet for my networked game because of it's raw power and portability. I'm not sure if it has a C# binding though.
Also, I recommend asynchronous.
But I can't help you with the C# specifics :v:
[editline].[/editline]
ninja'd
I just remembered that you can use $ for variable names
[img]http://i.imgur.com/Au2Rc.png[/img]
wonderful
Lidgren.Network is an excellent game networking library for C# games, does a lot of the heavy lifting for you in the lower-level areas.
[QUOTE=ief014;39264309]I just remembered that you can use $ for variable names
[IMG]http://i.imgur.com/Au2Rc.png[/IMG]
wonderful[/QUOTE]
char *$$$, not char* $$$.
[QUOTE=SiPlus;39264409]char *$$$, not char* $$$.[/QUOTE]
That's just a matter of taste.
[QUOTE=SiPlus;39264409]char *$$$, not char* $$$.[/QUOTE]
ew
[QUOTE=SiPlus;39264409]char *$$$, not char* $$$.[/QUOTE]
nobody* cares
[editline]18th January 2013[/editline]
[QUOTE=Exl;39257254]Is [url=http://cx-freeze.sourceforge.net/]cx_Freeze[/url] an option?[/QUOTE]
I'm not entirely sure, because it looks to me that programs made with cx_Freeze still need a Python interpreter installed to work, and if that's the case then I might as well just only ship the raw Python files.
I could be stupid though, I'll probably dust off my XP VM and see how cx_Freeze works that way.
[QUOTE=SiPlus;39264409]char *$$$, not char* $$$.[/QUOTE]
Doesn't matter as long as machine can read it.
[QUOTE=account;39250836]
I'm also trying to get rid of weird edge cases, like empty arrays and objects. Someone wanna give me some really weird input so I can test?[/QUOTE]
The facepunch ticker API
[editline]18th January 2013[/editline]
Redid most of my library. I'm going to start working on making the events threaded later.
[img]http://i.imgur.com/KtsJF.gif[/img]
[editline]18th January 2013[/editline]
Does anyone know a good screen recorder that can export webm etc and a proper place to host it so that I can embed it?
[QUOTE=Darkwater124;39264200]Asynchronous if you don't want your game to freeze every 50ms.
TCP if you're lazy, UDP if you want speed.[/QUOTE]
There are UDP libraries which provide reliable data transport, like ENet. You would use unreliable transport for game states, and reliable transport for small things like chat messages.
[QUOTE=SiPlus;39264409]char *$$$, not char* $$$.[/QUOTE]
one day your advice will be useful
one day
[img]http://i.imgur.com/YPBl4.gif[/img]
so instead of working on something that actually matters like map systems, i decided "i'm gonna make some text scroll."
Haven't worked on this in like a week due to me not wanting to deal with collision detection. Luckily I got that sorted and now I also added ceilings, and finally you can walk around a room instead of staring into space :L
Still waiting for my bad texture loading practices to bite my fps in the ass, right now i am loading up the spritesheet for every peice and then cropping it every frame, I wanted to just have a global spritesheet for the dungeon objects and crop the image data into sprites and use those but... until the game starts slowing down it's not a huge priority. I don't expect the maps to be that big anyway.
Anybody have any tips for me (I know the wall texture is ass, that is just temporary. I like the hud and floor texture though.)
[IMG]https://dl.dropbox.com/u/43867531/gameinprogress7.png[/IMG]
[QUOTE=Hypershadsy;39264113]I had an idea for a Bomberman type game. But before I do that, I need to know how to do networking.
So, networking and XNA devs, what do you use?
asynchronous or synchronous?
TCP or UDP?
System.Net.Sockets.Socket?[/QUOTE]
TCPClients are really easy in .NET and somewhat fun.
Christ, I passively lurk these threads seeing all of the wonderful things you guys create. It's amazing and makes me so jealous. I, like I'm sure you all are, am a very passionate gamer and would love to design games but I feel like I don't/wouldn't have the mental fortitude to learn how to and eventually create games.
Right now in life it feels way too risky (not in a "i'm going to follow my dream" sense) for me to take up a course in game design or programming, I have limited years of funding before I need to start paying for courses my self and I already wasted two years of my life and funding on a music course that left me with no practical qualifications.
Is it possible or even sensible to take up hobby programming instead of committing years of my life/funding to a course that I may or may not do well in? I'm an advocate for self-learning but it seems some topics are a lot more tricky to teach yourself. Do you guys have any examples of games that have come from someone who has taught themselves?
I feel I'd be much more inclined to make 2D or 2.5D games or gimmicks like creating a "modern day" Doom game. I like niche and different art directions as apposed to say much more gargantuan systems in comparion like full 3d environments or FPS/RTS/MMO type games.
Would you suggest it? Where would one even start?
I'm sure you guys get posts like this at least once a day but I'd like to see your input because you all impress me so much and I'd imagine the members of this forum are generally like-minded when it comes to the joy of gaming and, well for you guys.. making them..
(Also I'm aware this isn't specifically a *gaming* programming thread but I guess we're all on this site for one gaming-related reason or another, right?)
[QUOTE=LasGunz;39265250]Christ, I passively lurk these threads seeing all of the wonderful things you guys create. It's amazing and makes me so jealous. I, like I'm sure you all are, am a very passionate gamer and would love to design games but I feel like I don't/wouldn't have the mental fortitude to learn how to and eventually create games.
Right now in life it feels way too risky (not in a "i'm going to follow my dream" sense) for me to take up a course in game design or programming, I have limited years of funding before I need to start paying for courses my self and I already wasted two years of my life and funding on a music course that left me with no practical qualifications.
Is it possible or even sensible to take up hobby programming instead of committing years of my life/funding to a course that I may or may not do well in? I'm an advocate for self-learning but it seems some topics are a lot more tricky to teach yourself. Do you guys have any examples of games that have come from someone who has taught themselves?
I feel I'd be much more inclined to make 2D or 2.5D games or gimmicks like creating a "modern day" Doom game. I like niche and different art directions as apposed to say much more gargantuan systems in comparion like full 3d environments or FPS/RTS/MMO type games.
Would you suggest it? Where would one even start?
I'm sure you guys get posts like this at least once a day but I'd like to see your input because you all impress me so much and I'd imagine the members of this forum are generally like-minded when it comes to the joy of gaming and, well for you guys.. making them..
(Also I'm aware this isn't specifically a *gaming* programming thread but I guess we're all on this site for one gaming-related reason or another, right?)[/QUOTE]
Learning how to make games using something like Love2D would probably be a good starting point if you've not programmed before.
I'm sure plenty of us here are self-taught, by the time I had my first subject involving programming I had been doing it for 4-5 years. I can't point you at anything in particular since it depends on what you want to do, but learning Lua and using Love2D would be a good place to start looking.
hobby programming is easy. you just need an extremely angry man always telling you the flaws in your programs. so if you somehow have a shred of self esteem after all the verbal abuse you strive to get better and better until you became an absolute nazi when it comes to your own code and its quality.
easy!
[QUOTE=Shotgunz;39265336]hobby programming is easy. you just need an extremely angry man always telling you the flaws in your programs. so if you somehow have a shred of self esteem after all the verbal abuse you strive to get better and better until you became an absolute nazi when it comes to your own code and its quality.
easy![/QUOTE]
I've been wondering why nobody else has joined my programming classes...
[QUOTE=Jookia;39265344]I've been wondering why nobody else has joined my programming classes...[/QUOTE]
"This shit isn't accepting a UTF-32 string! F!"
Well thanks for the input guys :) is Love2D something you'd start literally from scratch with and learn or is there some sort of fundamentals or bare bones understanding I'd need to learn about game programming before hand?
[QUOTE=SiPlus;39264409]char *$$$, not char* $$$.[/QUOTE]
The cringe never ends.
[QUOTE=LasGunz;39265250]
Is it possible or even sensible to take up hobby programming instead of committing years of my life/funding to a course that I may or may not do well in? I'm an advocate for self-learning but it seems some topics are a lot more tricky to teach yourself. Do you guys have any examples of games that have come from someone who has taught themselves?[/QUOTE]
Sure, I've had zero education and I didn't even finish school but I feel I have the skill set to pretty much make any game I want now.
Just start programming instead of asking questions, you'll start to see how easy it is.
I don't have many questions on just.. where to start.. I understand I can go with Love2D and Lua and that's helpful to know which platform to start but.. I don't know how to even go about learning to use it. Their wiki has documentation on all of the things it [i]can[/i] do and even explanations of their use.. but that doesn't really teach me how to string it all together into something that is.. a thing :P
Just read [url=https://love2d.org/wiki/Getting_Started]this[/url] first and then [url=https://love2d.org/wiki/Tutorial:Hamster_Ball]this[/url]. Then you can pretty much start adding more stuff to the code to see what works and what doesn't. There are [url=https://love2d.org/wiki/Category:Tutorials]plenty of tutorials[/url] on the wiki.
[QUOTE=LasGunz;39265886]I don't have many questions on just.. where to start.. I understand I can go with Love2D and Lua and that's helpful to know which platform to start but.. I don't know how to even go about learning to use it. Their wiki has documentation on all of the things it [i]can[/i] do and even explanations of their use.. but that doesn't really teach me how to string it all together into something that is.. a thing :P[/QUOTE]
Maybe you should start with something like:
[url]http://www.udacity.com[/url]
[url]http://www.codecademy.com[/url]
Those have some basic courses on computer programming. It isn't games but you need to understand the basics first.
Sorry, you need to Log In to post a reply to this thread.
