Problem set 23

This problem set requires you to use this web site that has a neural network simulator. The options we will be fiddling with are the data sets, the features, the number of hidden layers, and the number of hidden nodes per layer.
  1. Use gaussian data, the first two features, 1 hidden layer with 1 neuron. Roughly how many epochs does it take to reach a test loss of 0?
  2. Change to the xor data. After 2,000 epochs, what is the Test loss? Is it correctly classifying most of the data? Does it look like it is getting much better?
  3. What is the minimum number of hidden neurons required to quickly (epochs < 1,000) get nearly all the data right (test loss < 0.04) two thirds of the time?
  4. Switch to the circle data. Does your current network work well on that data? How many epochs does it take to get a test loss less than 0.01?
  5. Switch to the spriral data. Is your network working after 2,000 epochs? Try 8 hidden nodes. What is the test loss after 2,000 epochs. How many weights are in this network?
  6. Try 2 hidden layers of of 4 nodes each. What is the test loss after 3,000 epochs? How many weights are in this network?
  7. Go full network, make both hidden layers have 8 nodes. What is the test loss after 2,000 epochs? How many weights are in this network? Try this 3 times and make a screenshot of the best one.
  8. Try going deep. Make a network with 6 hidden layers, with 3 nodes each. What is the test loss after 2,000? How many weights are in this network?
  9. Using the other features, design a network that gets a test loss less that 0.1 within 2,000 epochs. Take a screen shot. How many weights does it have?