PSet 23: Neural Networks
(videos: All Neural Network vids)
This problem set requires you to use a neural network simulator website. The options we will be fiddling with are the data sets, the features, the number of hidden layers, and the number of hidden nodes per layer.
Select gaussian data on the left, turn on just the first two features, use 1 hidden layer with 1 neuron. Roughly how many epochs does it take to reach a test loss of 0?
Change to the xor data with the same architecture from (1). After 2,000 epochs:
What is the Test loss?
Is it correctly classifying most of the data?
Does it look like it is getting much better?
Stick with the xor data, still 2 inputs and one neuron in the hidden layer. What is the minimum number of hidden neurons required to quickly (epochs < 1,000) get nearly all the data right (test loss < 0.04) two thirds of the time? Run it several times to make sure.
Switch to the circle data (not spiral).
Does your current network work well on that data?
How many epochs does it take to get a test loss less than 0.01?
Switch to the spriral data.
Is your network working after 2,000 epochs?
Try 8 hidden nodes. What is the test loss after 2,000 epochs?
How many total weights are in this network of 2 inputs, 8 hidden nodes, and 2 outputs?
Try 2 hidden layers of 4 neurons each.
What is the test loss after 3,000 epochs?
How many weights are in this network?
Go full network, make both hidden layers have 8 nodes.
What is the test loss after 3,000 epochs?
How many weights are in this network?
Try this 3 times and make a screenshot of the best one. Paste it here.
Try going deep. Make a network with 6 hidden layers, with 3 nodes each.
What is the test loss after 3,000?
How many weights are in this network?
Using the other features, design a network that gets a test loss less that 0.1 within 2,000 epochs. Take a screen shot. Paste it here. How many weights does it have?
When finished, work on your Frogger project.