poster
2019-12-14, 20:13
我正在尝试在MATLAB中实现“ 随机梯度下降 (http://en.wikipedia.org/wiki/Stochastic_gradient_descent) ”。我完全遵循了算法，但是对于预测/拟合函数，我得到了非常大的w（分量）。我的算法有误吗？
算法： https://i.stack.imgur.com/SH3mZ.png
x = 0:0.1:2*pi // X-axis n = size(x,2); r = -0.2+(0.4).*rand(n,1); //generating random noise to be added to the sin(x) function t=zeros(1,n); y=zeros(1,n); for i=1:n t(i)=sin(x(i))+r(i); // adding the noise y(i)=sin(x(i)); // the function without noise end f = round(1+rand(20,1)*n); //generating random indexes h = x(f); //choosing random x points k = t(f); //chossing random y points m=size(h,2); // length of the h vector scatter(h,k,'Red'); // drawing the training points (with noise) %scatter(x,t,2); hold on; plot(x,sin(x)); // plotting the Sin function w = [0.3 1 0.5]; // starting point of w a=0.05; // learning rate "alpha" // ---------------- ALGORITHM ---------------------// for i=1:20 v = [1 h(i) h(i).^2]; // X vector e = ((w*v') - k(i)).*v; // prediction - observation w = w - a*e; // updating w end hold on; l = 0:1:6; g = w(1)+w(2)*l+w(3)*(l.^2); plot(l,g,'Yellow'); // drawing the prediction function
回答：
如果您使用太大的学习率，SGD可能会有所不同。
升学率应收敛到零。
更多&回答... (https://stackoverflow.com/questions/5117337)
算法： https://i.stack.imgur.com/SH3mZ.png
x = 0:0.1:2*pi // X-axis n = size(x,2); r = -0.2+(0.4).*rand(n,1); //generating random noise to be added to the sin(x) function t=zeros(1,n); y=zeros(1,n); for i=1:n t(i)=sin(x(i))+r(i); // adding the noise y(i)=sin(x(i)); // the function without noise end f = round(1+rand(20,1)*n); //generating random indexes h = x(f); //choosing random x points k = t(f); //chossing random y points m=size(h,2); // length of the h vector scatter(h,k,'Red'); // drawing the training points (with noise) %scatter(x,t,2); hold on; plot(x,sin(x)); // plotting the Sin function w = [0.3 1 0.5]; // starting point of w a=0.05; // learning rate "alpha" // ---------------- ALGORITHM ---------------------// for i=1:20 v = [1 h(i) h(i).^2]; // X vector e = ((w*v') - k(i)).*v; // prediction - observation w = w - a*e; // updating w end hold on; l = 0:1:6; g = w(1)+w(2)*l+w(3)*(l.^2); plot(l,g,'Yellow'); // drawing the prediction function
回答：
如果您使用太大的学习率，SGD可能会有所不同。
升学率应收敛到零。
更多&回答... (https://stackoverflow.com/questions/5117337)