MATLAB feedforwardnet using gradient descent fails the first time

[poster]

I am training a feedforwardnet with gradient descent traingd as backpropagation algorithm to predict times table.

X = [repmat([1:10]', 10, 1) repelem([1:10]', 10)];

y = X(:, 1) .* X(:, 2);



net = feedforwardnet(8);                % Create a neural network with 8 neurons in the hidden layer

net.layers{1}.transferFcn = 'logsig';   % Hidden layer activation function set to logsig

net.trainFcn = 'traingd';               % Set backpropagation algorithm to gradient descent

net.divideParam.trainRatio = 0.6;

net.divideParam.testRatio = 0.2;

net.divideParam.valRatio = 0.2;

[net, TR] = train(net, X', y');         % Train the network

Whenever I first train the network the lowest validation error is too high as you can see below.

But then if I change my backpropagation algorithm to Levenberg-Marquardt trainlm, train the network and then switch back to gradient descent traingd and train again then my lowest validation error starts making sense.

Why is my training failing for the first time when I train it using gradient descent?

More answer...