MATLAB feedforwardnet using gradient descent fails the first time

[poster]

<p>I am training a <code>feedforwardnet</code> with gradient descent <code>traingd</code> as backpropagation algorithm to predict times table.</p>

<pre><code>X = [repmat([1:10]', 10, 1) repelem([1:10]', 10)];
y = X(:, 1) .* X(:, 2);

net = feedforwardnet(8); % Create a neural network with 8 neurons in the hidden layer
net.layers{1}.transferFcn = 'logsig'; % Hidden layer activation function set to logsig
net.trainFcn = 'traingd'; % Set backpropagation algorithm to gradient descent
net.divideParam.trainRatio = 0.6;
net.divideParam.testRatio = 0.2;
net.divideParam.valRatio = 0.2;
[net, TR] = train(net, X', y'); % Train the network
</code></pre>

<p>Whenever I first train the network the lowest validation error is too high as you can see below.</p>

<p><img src="https://i.imgur.com/xCsv0Jn.png" alt="Training with gradient descent"></p>

<p>But then if I change my backpropagation algorithm to Levenberg-Marquardt <code>trainlm</code>, train the network and then switch back to gradient descent <code>traingd</code> and train again then my lowest validation error starts making sense.</p>

<p>Why is my training failing for the first time when I train it using gradient descent?</p>

<p><img src="https://i.imgur.com/5xoUHdK.png" alt="Training with levenberg-marquardt"></p>

<p><img src="https://i.imgur.com/j0wK3Tj.png" alt="Training with gradient descent after switching from trainlm"></p>



More answer... (https://stackoverflow.com/questions/59040050/matlab-feedforwardnet-using-gradient-descent-fails-the-first-time)