A Little Weird » Blog Archive » Neural Networks pt. 1

A Little Weird

reality bizarres the standard

August 15th, 2009 by Sean Neural Networks pt. 1

Background

Artificial Intelligence (AI) is a really cool subject. It combines two things I love - programming, and the mind. It’s interesting to read about psychology and biology… but for me, programming is more real. A psychologist or biologist studies the mind/brain. An AI programmer attempts to create a mind/brain.

In AI, a popular technique is to use something called a Neural Network. Basically, the programmer first attempts to write code to mimic a single neuron. Then, the programmer creates a network of these individual neurons, and watches them interact. This technique comes from the belief that our brain’s power comes from the billions of networked neurons inside of it.

AI programmers first came up with the idea of making a digital Neural Network from talking to biologists. After all, if you want to build a digital brain, why not talk to people who study real brains? Biologists taught programmers how neurons actually worked - programmers took the concept, and create a simplified digital counter-part.

There is another way to arrive at the conclusion that a Neural Network is a good idea, though. We can arrive there just using logic and math, without any knowledge of how real brains work. Granted, this line of reasoning only became clear in hindsight - but it explains why evolution might have naturally created the brain in the first place:

The Neural Network’s Purpose

A Neural Network is awesome at one thing - detecting relationships.

Imagine you have a series of numbers… 12, 4, 0, 4, 16, 3, 7, 6, 9, 11. Just with that series of numbers, you have no clue about any meaning behind it. It could be a random sequence, or some parts might be random, but other parts might have meaning. Some formula might determine the numbers. Or it might represent money, or time, or butterflies, or word counts, etc. As it stands right now - you simply don’t know.

Now imagine I feed you a stream of numbers, like so:

1. 12, 4, 0, 4, 16, 3, 7, 6, 9, 11

2. 13, 6, 0, 3, 18, 2, 7, 4, 10, 12

3. 14, 8, 0, 2, 16, 1, 7, 2, 11, 13

4. 15, 10, 0, 1, 10, 0, 7, 0, 12, 14

If you’re clever, you’ll notice there are a lot of patterns in that stream. The first number always increases by one, the second increases by two, the third is always zero, the fourth decreases by one, the fifth is the second number times the fourth number, etc.

You can see the pattern because you have a brain, and I just told you what the patterns are. Imagine you are a computer, with zero knowledge, and your job is to figure out the pattern from scratch. How would you go about it? How would you conclude that the fifth number is the second times the fourth?

Since you have no prior knowledge, what you have to do is try every combination. Is the first number the second times the third? No. Is the first number the second times the fourth? No. Is the first number the second times the fifth? No. Eventually, while trying everything out, you find that the fifth number is the second times the fourth.

The only way the computer can detect a pattern is by trying every possible combination, and see if it finds anything. In fact, this is what humans do too - if you look at a sequence of numbers, and attempt to find a pattern, you’ll sit there and try different things out… searching for something that makes sense.

Creating the Network

Why create a network of neurons? You can think of a neuron as a little detector. It takes in input, and spits out a “yes” or “no”. Then, more neurons take in a collection of yes’s and no’s, and spit out their own yes or no. So there’s a bunch of these yes’s and no’s flying around.

The part that makes this work is that a neuron learns which neighboring neurons it should listen to, and which ones it should ignore. You can think of the neuron as being networked with the neighbors it listens to. Likewise, there are neighbors that listen to it. This creates a web of neurons.

Network Complexity

Back to our random number example - you need a neuron at every spot you want to detect a relationship. Is the first number the second times the third? Stick a neuron there. You’ll need a network of neurons capable of performing multiplication, but you’ll need one neuron at the top that fires off a “yes” or “no” if the first number is the second times the third.

We have a lot of different relationships we want to test… so we’ll need a lot of neurons. And what if we want to test really complicated relationships? For example, maybe we want to test the fifth number is the second, plus the third, times the sixth. We’ll need a neuron for that.

So, the more complicated the relationships, the more neurons we’ll need. Really, there’s no limit… because there could be an insanely complicated relationship behind two numbers. If we don’t have enough neurons to detect the relationship, then our neural network won’t be able to figure it out.

Conclusion

There is a lot of information on Neural Networks, and I’ve glossed over a lot of the details. The important point to recognize is that you can use a network of neurons in order to detect relationships. The more neurons, the more complicated the relationships it can detect. Each individual neuron performs a simple task, but when networked intelligently (from learning over time which neighbors to listen to, and which to ignore), the network is capable of extracting meaning from a stream of input.

Stay tuned for part two…

This entry was posted on Saturday, August 15th, 2009 at 10:11 am and is filed under Consciousness. You can follow any responses to this entry through the RSS 2.0 feed. You can skip to the end and leave a response. Pinging is currently not allowed.

### 2 Responses to “Neural Networks pt. 1”

1. f3licks Says:
   August 18th, 2009 at 11:06 pm

   Do the neurons themselves learn which neighbor to ‘listen’ to, or is that just your brain creating an effective network.

2. runical Says:
   August 22nd, 2009 at 4:34 am

   that is pretty interesting. So the digital neurons must learn to which neurons they have to listen. however, can a neuron answer multiple questions or can it only detect 1 specific thing? If they don’t, you need a lot of those things…

   Looking forward to part 2.

### Leave a Reply

You must be logged in to post a comment.

- ## Pages

  - Welcome!
  - Blog
  - Forums
  - Games
    - Emergence Game
    - Geiger Game
    - Lottery Game
  - Contact

- ## Experiments

  - Lottery Project, Part 1
  - Lottery Project, Part 2

- ## Categories

  - Belief Systems (28)
  - Books (7)
  - Consciousness (4)
  - Conspiracy (5)
  - Dream Phenomenon (24)
  - ESP (17)
  - How-To (11)
  - Lottery (32)
  - Note to Self (21)
  - Personal Development (17)
  - Philosophy (31)
  - Random Stuff (11)
  - Science (26)
  - Subconscious (14)
  - Uncategorized (1)
  - Video (16)
  - Website (13)

- ## Archives

  - November 2011
  - November 2010
  - June 2010
  - February 2010
  - December 2009
  - November 2009
  - October 2009
  - August 2009
  - July 2009
  - May 2009
  - April 2009
  - March 2009
  - February 2009
  - January 2009
  - November 2008
  - October 2008
  - September 2008
  - August 2008
  - July 2008
  - June 2008
  - May 2008
  - April 2008
  - March 2008
  - February 2008
  - January 2008
  - December 2007
  - November 2007
  - October 2007
  - September 2007
  - August 2007
  - July 2007
  - June 2007
  - May 2007
  - April 2007

- ## Links

  - A State of Mind
  - Dreaming to Infinity
  - Monroe Institute on YouTube
  - Shifted Perspectives
  - UC Berkeley on YouTube

- ## Meta

  - Login
  - Valid XHTML

All Content, Images, Video, Text, and Software is © Copyright 2007-2009 Sean Connelly and their respective authors. All Rights Reserved.
Best viewed with Firefox, with minimum resolution of 1024 x 768. Microsoft can suck my balls.
A Little Weird is proudly powered by WordPress - Entries (RSS) and Comments (RSS).