Simple Neural Network
Introduction
This is a simple neural network to solve classic non-linear XOR (exclusive) problem.
Problem
XOR problem is a classic non-linear classification problem where you take 2 binary inputs and output either 0 or 1 depending on the combination of input. Here, we want to output the correct classification of XOR where:
| Input 1 | Input 2 | Output |
|---|---|---|
| 0 | 0 | 0 |
| 0 | 1 | 1 |
| 1 | 0 | 1 |
| 1 | 1 | 0 |
Input Matrix X
| Input 1 | Input 2 |
|---|---|
| 0 | 0 |
| 0 | 1 |
| 1 | 0 |
| 1 | 1 |
Output Matrix y
| Output |
|---|
| 0 |
| 1 |
| 1 |
| 0 |
Solution
To solve XOR classification problem, we will use the neural network algorithm which is universal non-linear function approximator to correctly classify the XOR output. We will implement the simplest architecture to solve this problem.
Architecture
Input Layer
2 input neurons
Hidden Layer
2 hidden neurons
Output Layer
1 output neuron
Components
Feedforward
Activation Function
- Sigmoid Function s(x) = 1 / 1 + e^-x
Loss Function
- Squared Error E = truth - prediction
Backpropagation
Optimization Algorithm
- Gradient Descent W += Delta
Future Update
- Create a tutorial explaining underlying mathematical concepts, algorithms and code.
- Design and implement OOP neural network.
- Design Neural Network to solve Image Classification Problem
Download
Resources
Neural Networks and Deep Learning
A Neural Network in 11 lines of Python