我已经实现了反向传播算法来训练我的神经网络 . 它完美地解决了AND＆OR，但是当我尝试训练解决XOR时，总误差非常高 .
XOR网络的网络拓扑结构是：输入层有2个神经元，隐藏层有2个神经元，输出层有一个神经元 .
我使用sigmoid作为激活函数，加权和作为输入 .
以下是我负责反向传播的代码部分：
protected void updateOutputLayer(double[] outputErr)
{
double delta;
Neuron neuron;
double errorDerivative;
for ( int i=0;i<this.getNeuralNetwork().getOutputLayer().getSize();i++)
{
neuron=this.getNeuralNetwork().getOutputLayer().getAt(i);
errorDerivative=neuron.getTransferFunction().getDerivative(neuron.getNetInput());
delta=outputErr[i]*errorDerivative;
neuron.setDelta(roundThreeDecimals(delta));
// now update the weights
this.updateNeuronWeights(neuron);
}
}
protected void updateHiddenLayerNeurons()
{
List<Layer> layers=this.network.getLayers();
Layer currentLayer;
double neuronErr;
for ( int i=layers.size()-2;i>0;i--)
{
currentLayer= layers.get(i);
for (int j=0;j<currentLayer.getSize();j++)
{
neuronErr=calculateHiddenLayerError(currentLayer.getAt(j));
currentLayer.getAt(j).setDelta(neuronErr);
this.updateNeuronWeights(currentLayer.getAt(j));
}
}
//System.out.println("*****************************************");
}
protected double calculateHiddenLayerError(Neuron node)
{
List<Connection> outputCon= node.getOutputConnections();
double errFactor=0;
for (Connection outputCon1 : outputCon) {
//System.out.println("output od dst: "+outputCon1.getDst().getOutput());
// System.out.println("w dst: "+outputCon1.getWeight());
//System.out.println("in CalcErr Factor err: "+outputCon.get(i).getDst().getError()+" w: "+outputCon.get(i).getWeight());
errFactor += outputCon1.getDst().getDelta() * outputCon1.getWeight();
}
double derivative= node.getTransferFunction().getDerivative(node.getNetInput());
return roundThreeDecimals(derivative*errFactor);
}
public void updateNeuronWeights(Neuron neuron)
{
double weightChange;
double input, error;
for (Connection con: neuron.getInConnections())
{
input=con.getInput();
// System.out.println("input: "+input);
error = neuron.getDelta();
weightChange=this.learningRate*error*input;// error here is : output error * error derivative
con.setWeight(roundThreeDecimals(con.getWeight()+weightChange));
}
// now update bias
if(neuron.isBiasUsed())
{
//System.out.println("old bias: "+neuron.getBias());
double biasChange=neuron.getBias()+neuron.getDelta()*this.learningRate;
//System.out.println("new bias: "+biasChange);
neuron.setBias(roundThreeDecimals(biasChange));
}
}
我使用的学习率在[0.01,0.5]范围内 . 谁能告诉我我的代码有什么问题？
1 回答
TL;DR: 您应该使用retropropagation以与学习权重相同的方式更新偏差 .
当然，与OR或AND相比，偏差在学习XOR方面起着重要作用（见Why is a bias neuron necessary for a backpropagating neural network that recognizes the XOR operator?） . 因此，偏见可能是罪魁祸首 .
你说
I'm using sigmoid as my activation function, and weighted sum as input. You need a bias than can be learned in the very same way that weights are learned. 注意：在应用激活函数之前，应在求和中添加偏差 .