gBrain: Need moar neurons - General

Ok, so here’s the skinny: gBrain is a self-developing operating system for Garry’s Mod drones. It has a decisions side (which uses neural networks), an actions side (which uses genetic algorithms), and a central core that moderates between them. The neurons in the decisions side use the current known state of the drone and the detectable input values of the world around it to select an action to perform, the actions side uses its genetic algorithms to determine the best course of movement to perform within the actions its already undertaking, the central core determines which one is more certain its idea is the best and goes with them. Whilst this is heavily entwined and intended for Garry’s Mod, it’s predominantly a mathematical project, more programming exercise than gameplay challenge. So, why am I posting this here instead of Wiremod, the more maths-ready forum? Well, two reasons.[list] [*]The first being that the lack of co-operation between Wiremod and Facepunch, arguably the two largest gmod communities, baffles me. I’m very certain a lot of very awesome stuff could result if the two worked together more harmoniously. [*]The second is that, well, the maths in the decisions side at the moment have reached a conundrum that is sorta baffling me too.[/list] Let me explain:[list] [*]A neural network is made of individual “neurons”, like the brain, arranged in layers. [*]The lowest layer is connected to the input values, and every layer afterwards is connected to every neuron in the previous layer (usually ending with 1 neuron at the highest layer). [*]Each input in a neuron is assigned an initially-random number, called a “weight”, which it multiplies that input value by, sums up the different results, puts the total through a sine function (such as 1/1-exp(-Sum)) and outputs itself to the next layer. [*]To get the network to respond the right way to a given set of input values, such as In1=0 and In2=1 so Target=1, you need to alter the weight values of all the neurons until the final output of the network meets the target value.[/list] Understandably, doing so by hand is time-consuming and high-stress, so neural networks work most of the time at doing so themselves; which means mathematical formulae need to be made to determine how the weights change. gBrain uses backpropagation networks, meaning you send your input set through the network, find the output value, then work backwards through all the neurons until you reach the input layer and start over. The way each neuron’s weights are changed depends on what the neurons ahead of them need to receive to output what they need, so you need two formulae to work things out, one to find the number a given input on a neuron needs, and one to determine how to change each weight in the input neuron connected to it to give that number. So if you’ve read all that and still have some vague idea about what I mean, let’s move on to my formulas as of right now, which are certainly working but are fairly unreliable and tend to degrade after a certain point. First up, figuring out what each neuron prior to the final layer needs to output. (Target*(Input/TInput))/Weight[list] [*]Target = The value the input needs for the proper contribution (For the final neuron, this is the target output of the network) [*]Input = The output value of the neuron associated with the weight [*]TInput = The total input sum for the current neuron, before sine function again [*]Weight = The weight associated with the predecessor neuron (The value by which that neuron's output is multiplied in the current neuron)[/list]Input/TInput is essentially just the percentage of the total input that is represented by the input, or what portion of the target value is represented by it. Dividing it by the weight gives us the input value we need. Now, the formula for how we change the weights Weight -= ((NeedOut*(WeightOut/PrevOut))/Weight)*Learn[list] [*]NeedOut = The necessary output from the current neuron to the next (see above formula) [*]WeightOut = The result of the last stimulus to the weight multiplied by it (What the weight last output into the neuron, basically) [*]PrevOut = The previous total of the neuron's weight results, prior to the sine function [*]Learn = The Learning rate, a percentage, which moderates how much each weight can change so that it doesn't overshoot the target [*]Weight = The weight in question[/list]WeightOut/PrevOut gives us the percentage of the weight's previous calculation that existed in the total output of the neuron, which multiplied by the necessary output gives us the value we need from it, and /Weight gives us the number we need prior to multiplication. It’s a minus because my experiments in Excel seem to show that to be more successful more often than addition, though far less stable. So, if there's anyone here with some mathematical ability that can lend me a hand, I'd much appreciate it.

So are you trying to code physics for it or what?

Far from physics, this is AI programming in lua. Kinda neat, really.

Not LUA, E2 code, but both use the same maths. The problem at the moment is that the neurons either never reach the target or do so and then go away from it again, often in an erratic pattern. EDIT: Okay, the net output stabilises quite significantly if, instead of using the direct needed input, we use the separation of the existing input from the needed input (Meaning, add -PrevOut to the end of the first formula). However, there is a problem still, in that for some entirely unknown reason, these formulae can [i]only[/i] move towards 1, not 0 or -1, which is a rather substantial fault for it's purpose. If it actually does try to reach -1, it seems to get stuck around -2 instead. The 0 I can understand due to the lack of a value to percentalise, but the -1 is a mystery to me.

Interesting.

You should make it auto-detect wrong sections

I wrote a 10 page research paper on Neural Networks last semester (and then created a blackjack game in which the computer does not know the rules but learns them overtime). I'm not quite sure what you exactly are trying to teach the network to do (or are you just trying to implement it?). Anyways, take a look at [URL="http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-index.html"]http://fbim.fh-regensburg.de/~saj39122/jfroehl/diplom/e-index.html[/URL]. Even if you don't know java, you can get a feel for how implementing neural networks works. Fun Fact: once the network is trained to recognize what you want it to, it is impossible to determine how it is reaching it's conclusion. Edit: It should also be noted that the math behind most neural networks is an industry secret. Your best bet is to look for an open source project and implement it via a dll or something.

I don't know what it is, but it sounds cool...

If you can get this to work... you would have made Freud a very proud man.

Wrong sextion

Lua is definitely not an AI language and 'intelligent' actions cannot be replicated in Lua

I reiterate: This is not being produced in LUA, it is an Expression 2 project. This is however irrelevant to the assistance I'm asking for, which is entirely mathematic.

-Snip, Misread- Can you explain your objective more in depth, please?

You're not the only person working with neural networks at the moment. ;) If you want to discuss it or team up add me on steam, my id is flie171.

obligatory-SKYNET-post. Seriously, sounds awesome. If this means we can see Cleverbot(bad example, but meh) in Gmod, would be awesome.

[QUOTE=ExplodingGuy;22873475]-Snip, Misread- Can you explain your objective more in depth, please?[/QUOTE] The objective is to produce a system that will manipulate control variables when connected to a chassis in order to determine autonomically the most effective manner for the chassis to function. In brief: To create a system that can effectively control any robot body it's connected to. The neurons are the issue currently at hand. We've managed forward-propagation neurons on Wiremod, but we're trying for backpropagation, which is throwing out some issues at the moment as described above. The neurons are intended to make the snap decisions and the moment-to-moment decisions, by figuring out what it's reactions should be from good/bad impulses and then rewiring the associated artificial neural networks to properly respond to them. The other side of the gBrain is genetic algorithms, segments of code that undertake sequences of change in the control variables like the way a cell does what the DNA inside of it does. Occasional mutation and recombination based on the degree of effectiveness of each string produces a selection of increasingly-effective sequences, tied to specific pre-defined goals and actions, like standing, walking, searching, running, etc. The neurons determine which action among the selection is to be performed at a given time, as well as whether to stop doing so, whilst such an action is under way the algorithm side also outputs it's own value of certainty which the central core decides based on the certainty of the neural network. In short: The neurons decide to walk, the algorithms say how to. Because it's all in code, the attached chassis is nebulous and immaterial, so any chassis will do, be it a car or a mech or whatever.

If you end up creating robots that get smarter as their numbers grow, I want nothing to do with it. In case the robots gain sentience and decide to rebel against you, I don't want to be on the same server at that point.

You do know there is a GMod section for this kind of shit, right kid? Anyway, neural networks can end up getting pretty confusing if you look too far into it.

[QUOTE=Lyinginbedmon;22907339]-gBrain Stuffs-[/QUOTE] Okay, let me iterate, You need it to recall previous steps, but however, it recalls the incorrect step. Is this right?

[QUOTE=Lyinginbedmon;22907339]The objective is to produce a system that will manipulate control variables when connected to a chassis in order to determine autonomically the most effective manner for the chassis to function. In brief: To create a system that can effectively control any robot body it's connected to. The neurons are the issue currently at hand. We've managed forward-propagation neurons on Wiremod, but we're trying for backpropagation, which is throwing out some issues at the moment as described above. The neurons are intended to make the snap decisions and the moment-to-moment decisions, by figuring out what it's reactions should be from good/bad impulses and then rewiring the associated artificial neural networks to properly respond to them. The other side of the gBrain is genetic algorithms, segments of code that undertake sequences of change in the control variables like the way a cell does what the DNA inside of it does. Occasional mutation and recombination based on the degree of effectiveness of each string produces a selection of increasingly-effective sequences, tied to specific pre-defined goals and actions, like standing, walking, searching, running, etc. The neurons determine which action among the selection is to be performed at a given time, as well as whether to stop doing so, whilst such an action is under way the algorithm side also outputs it's own value of certainty which the central core decides based on the certainty of the neural network. In short: The neurons decide to walk, the algorithms say how to. Because it's all in code, the attached chassis is nebulous and immaterial, so any chassis will do, be it a car or a mech or whatever.[/QUOTE] As neat as it sounds, that approach is flawed in my opinion; You can't have a 'super network' that controls any configuration. From what you wrote, it seems like the network has no way of establishing a grip on the [i]immediate[/i] relationship of the sensory data it receives and the resulting output behaviour, this means that it can't associate inputs and outputs correctly to develop reflexes; a network that can't associate is a bad one. What you need is a system that 'grows' a specialised network to control specific configurations; or better yet, one that grows both. Using back-propagation is a bad choice here, what you clearly need is unsupervised learning, something that can improvise; yet you choose the worst tool available: back-propagation, which may come as a popular first choice for pattern recognition/data mining problems.. but is unsuitable for real-time active sensory applications (aside from being used as a sub-component of some logic systems), but right here, you simply don't have that which you need the most to produce a good back-prop. network: a good training dataset. This is why you shouldn't automatically throw back-propagation at any problem you're faced with, and why I hate its overuse in everything. Anyway, My advice is to apply a 'master' genetic encoding that represents both the network's wiring and the behavioural algorithms.. and evolve it as a whole, ditching back-propagation completely and switching to a [url=http://en.wikipedia.org/wiki/Spiking_neural_network]spiking neural network[/url] model; which are a much better alternative for what you're trying to do. Now use a fitness formula to sift through a random initial population; repeat till satisfied.

Yes, ideally I'd like it to figure out associations of inputs and outputs by itself, however as it stands that's one of the few things that actually has to be pre-determined before attachment by the player. Inputs are fed to the brain as a numerical array as are the outputs by which the actual control is utilised (So relations are X and Y inputs to "turn" action or N & O to Z (Creating two separate nets for + and -), and such). Each network works towards it's own target as determined by a data set that is part-player part-gBrain built. For example, you can pre-set Net7 to output 1 when it's first input is 37 and it's second input is 0. Depending on the pain/pleasure responses the brain can then build a more conclusive and shifting target set. The problem at the moment is that, using the formulae stated in the opening post, the neurons do work towards the proper outcome most of the time. However, occasionally they fail entirely, or more bizarrely reach the proper conclusion but then "degrade", falling away from the target value. Stability has recently been achieved with a few slight alterations to said formulae, so I'm sure the problem isn't so much the formulae as how they're laid out presently. Effectively the results are close enough for me to be certain they're on the right path, but far enough for them to be insufficient for gBrain's functioning. Also, unsupervised learning and back-propagation are two separate elements of neural networks. One is acquisition of a target output value, the other is a method of reaching that output value. Initial tests utilised forward-propagation, as well. Backpropagation is presently under investigation as an alternative, test results will ultimately determine whether it supercedes forward.

[QUOTE=Lyinginbedmon;22951742]Yes, ideally I'd like it to figure out associations of inputs and outputs by itself, however as it stands that's one of the few things that actually has to be pre-determined before attachment by the player. Inputs are fed to the brain as a numerical array as are the outputs by which the actual control is utilised (So relations are X and Y inputs to "turn" action or N & O to Z (Creating two separate nets for + and -), and such).[/QUOTE] I can't say I fully understand the limits behind the mechanics you describe, as my experience with gmod is, admittedly: nil. (I've never even played it) [QUOTE=Lyinginbedmon;22951742]Each network works towards it's own target as determined by a data set that is part-player part-gBrain built. For example, you can pre-set Net7 to output 1 when it's first input is 37 and it's second input is 0. Depending on the pain/pleasure responses the brain can then build a more conclusive and shifting target set.[/QUOTE] I see now, but the problem remains; for a network that can't associate, if the probability space is infinite; the training set has to match. Unless, the network is able to associate; in which case it'll figure out the underlying algorithm on its own, which fills the holes in the output spectrum. [QUOTE=Lyinginbedmon;22951742]The problem at the moment is that, using the formulae stated in the opening post, the neurons do work towards the proper outcome most of the time. However, occasionally they fail entirely, or more bizarrely reach the proper conclusion but then "degrade", falling away from the target value. Stability has recently been achieved with a few slight alterations to said formulae, so I'm sure the problem isn't so much the formulae as how they're laid out presently. Effectively the results are close enough for me to be certain they're on the right path, but far enough for them to be insufficient for gBrain's functioning.[/QUOTE] Yes, what you're describing is the issue of Overlearning (or more commonly, in the statistical domain what is called [url=http://en.wikipedia.org/wiki/Overfitting]Overfitting[/url]): Overlearning (in Neural Networks): When an iterative training algorithm is run, overfitting which occurs when the algorithm is run for too long (and the network is too complex for the problem or the available quantity of data). And that's why I suggested ditching supervised learning completely, try as you might, you simply can't gather enough training data for a network that can't associate; more often than not, there will always be that special situation that your training set did not account for. [QUOTE=Lyinginbedmon;22951742]Also, unsupervised learning and back-propagation are two separate elements of neural networks. One is acquisition of a target output value, the other is a method of reaching that output value.[/QUOTE] Unsupervised learning is the opposite of supervised learning, back-propagation is a method utilised in supervised learning, hence: you train the network, and back-propagation determines how the network adjusts its weights to minimize the error margin. What I suggested was using non-linear spiking neural networks, where the topology of the network itself contributes to the network's logic, to make this clearer, think of biological brains, those have no 'layers', instead they have specialised subsystems, and those can only be produced in two ways: by hand, or as the product of an evolutionary algorithm, of which the latter would be the ideal choice for your project, of course. Another alternative that doesn't require starting from scratch is switching to a [URL="http://en.wikipedia.org/wiki/Reinforcement_learning#Algorithms"]Reinforced learning algorithm[/URL], which closely resembles what you're trying to do to a certain extent, and stands in the middle of both approaches, albeit a bit harder to implement correctly. [QUOTE=Lyinginbedmon;22951742]Initial tests utilised forward-propagation, as well. Backpropagation is presently under investigation as an alternative, test results will ultimately determine whether it supercedes forward.[/QUOTE] I'm quite curious about the results you got back then and the ones you're getting now, did you record a video or something you could post perhaps?

[QUOTE=voodooattack;22955024]I see now, but the problem remains; for a network that can't associate, if the probability space is infinite; the training set has to match. Unless, the network is able to associate; in which case it'll figure out the underlying algorithm on its own, which fills the holes in the output spectrum.[/quote] Presently, the network cannot autonomically associate, the inputs of each network are predetermined by the user as an array of strings. If an effective and efficient way can be found to have the brain do so itself, I'll definitely be glad to incorporate it. [QUOTE=voodooattack;22955024]Yes, what you're describing is the issue of Overlearning (or more commonly, in the statistical domain what is called [url=http://en.wikipedia.org/wiki/Overfitting]Overfitting[/url]): Overlearning (in Neural Networks): When an iterative training algorithm is run, overfitting which occurs when the algorithm is run for too long (and the network is too complex for the problem or the available quantity of data). And that's why I suggested ditching supervised learning completely, try as you might, you simply can't gather enough training data for a network that can't associate; more often than not, there will always be that special situation that your training set did not account for.[/quote] Overfitting didn't occur with the forwardpropagation method, and I'm still unsure how it's happening with backpropagation at all. Near as I can tell, the modifications should always be [i]towards[/i] the goal value, yet they ripple and shake. This is the mathematical consternation that lead to this thread. [QUOTE=voodooattack;22955024]What I suggested was using non-linear spiking neural networks, where the topology of the network itself contributes to the network's logic. To make this clearer, think of biological brains, those have no 'layers', instead they have specialised subsystems, and those can only be produced in two ways: by hand, or as the product of an evolutionary algorithm, of which the latter would be the ideal choice for your project, of course. Another alternative that doesn't require starting from scratch is switching to a [URL="http://en.wikipedia.org/wiki/Reinforcement_learning#Algorithms"]Reinforced learning algorithm[/URL], which closely resembles what you're trying to do to a certain extent, and stands in the middle of both approaches, albeit a bit harder to implement correctly.[/quote] I'll certainly look into them, right now I'd rather resolve backpropagation before considering alternative approaches however (Once the options have been tested and confirmed as viable within the programming, we can properly compare and consider which to implement into the final model) [QUOTE=voodooattack;22955024]I'm quite curious about the results you got back then and the ones you're getting now, did you record a video or something you could post perhaps?[/QUOTE] Simply put, forwardpropagation showed an "infallible" (using the term lightly here...) progression over time towards the target values. However, it required a rather substantial time to adapt (which thus far seems to be significantly greater than backpropagation) to just 2 test sets. It always achieved it's goal in time, but backprop seems to require a fraction of the time for the same goal, [i]when it works[/i].

This should probably go in Garry's Mod section, you aren't going to get a lot of serious replies here.

[QUOTE=DiscoBiscut;23009509]This should probably go in Garry's Mod section, you aren't going to get a lot of serious replies here.[/QUOTE] 1. this has been said already 2. you need to learn to read, 70% of replies in this thread are serious and informative

As said repeatedly, the matters involved here are mathematical and implementational, only their final implementation will be Garry's Mod related. The only post here considering Garry's Mod intensely for this project is my opening post. It is not the focus of this thread.

Why would you code it in E2 when LUA would be far better for this? E2 would just make it slow.

Simply because LUA needs to be downloaded, E2 just needs arguably the most popular addon in Garry's Mod to be installed and reasonably up-to-date. Also because I have no clue how to begin starting to code LUA, so doing this in E2 will be faster.

Quick stop by to drop off this general-case neural network running code. There's nothing in here for generating or modifying the weights, this is just the bare bones "get the network doing something" code, it runs sequentially through all the neuron layers.[code]@persist Step #Neur = an array of all the neuron weights in a string form #Data = an array the inputs to be fed to the current neuron layer #Next = an array of the outputs from the current layer #Weights = an array of the current neuron's weights #Step = the neuron number at which the layer next changes, base2 Step=round(2^(log2(Neur:count())-1)) for(N=Neur:count(),0,-1){ #Layer changeover #Use of base2 halves the neurons in each layer #so eventually we hit 2 neurons and complete. if(N==Step){ Data=Next:clone() Next=array() Step=round(2^(log2(N)-1)) } #Neuron action Weights=Neur[N,string]:explode("+") T=0 for(M=1,Data:count()){ T+=Data[M,number]*Weights[M,string]:toNumber() } Next[N,number]=1/1-exp(-Sum) }[/code]Hopefully this gives people a hand in focusing on the weight modification portion of it, though I'm not entirely sure of the effectiveness of the layer changing code, it may or may not add a superfluous layer and it definitely only works with powers of 2. But hey, it's early in the day and I'm out of caffeine, I'm happy to have made this much off the top of my head.