[译]C ++中的反向传播人工神经网络
By robot-v1.0
本文链接 https://www.kyfws.com/ai/backpropagation-artificial-neural-network-in-c-zh/
版权声明 本博客所有文章除特别声明外,均采用 BY-NC-SA 许可协议。转载请注明出处!
- 14 分钟阅读 - 6577 个词 阅读量 0C ++中的反向传播人工神经网络(译文)
原文地址:https://www.codeproject.com/Articles/21171/Backpropagation-Artificial-Neural-Network-in-C
原文作者:Chesnokov Yuriy
译文由本站 robot-v1.0 翻译
前言
This article demonstrates a backpropagation artificial neural network console application with validation and test sets for performance estimation using uneven distribution metrics.
本文演示了一种反向传播人工神经网络控制台应用程序,其中包含使用不均匀分布指标进行性能评估的验证和测试集.
介绍(Introduction)
我想介绍我在医学数据分类和用于面部检测的CV库研究期间开发和使用的反向传播神经网络C ++库的基于控制台的实现:(I’d like to present a console based implementation of the backpropogation neural network C++ library I developed and used during my research in medical data classification and the CV library for face detection:) 具有皮肤和动作分析的人脸检测C ++库(Face Detection C++ library with Skin and Motion analysis) . The CodeProject已经有一些不错的文章,您可以参考他们的理论.在我的代码中,我使用Minmax,Zscore,Sigmoidal和Energy规范化来介绍输入层中作为输入数据预处理的必要功能.这些参数是从训练集中获得的,然后用于预处理每个输入向量以进行分类.该控制台支持训练数据随机分离,以在反向传播训练之前进行训练,验证和测试集.随机分离允许获得代表性的训练集,比较验证和测试零件的性能.验证集可通过估计该集的性能来防止过度拟合.在反向传播会话结束时,我保存了两个网络配置,即在验证集上性能最佳的网络配置和最后一个训练时期的配置.对于性能评估,我使用敏感性,特异性,正向预测性,负向预测性和准确性指标.为了在数据分布有偏差的情况下进行验证性能估计(例如,在我的人脸检测文章中,您可能会发现非人脸比人脸多得多,比率为19:1),我提供了几何平均值和F量度支持方案的指标.为了支持大量数据向量,我提供了基于文件映射的数据加载.这样一来,您就可以在1毫秒内将数百兆的训练数据映射到内存,并立即开始训练.对于相对少量的数据,可以使用文本文件格式.最后,控制台实现更易于使用,避免了GUI应用程序中的大量鼠标单击,并且可以使用批处理文件自动执行此过程,以选择正确的网络拓扑,验证和测试集的最佳性能,等等.(. There are some good articles already present at The CodeProject, and you may consult them for the theory. In my code, I present the necessary features as input data preprocessing in the input layer with Minmax, Zscore, Sigmoidal, and Energy normalization. These parameters are obtained from the training set, and then used for preprocessing every incoming vector for classification. The console supports training data random separation to train, validation, and test sets before backpropagation training. Random separation allows to obtain a representative train set comparing performance on validation and test parts. A validation set is useful for preventing over-fitting by estimating the performance on that set. At the end of the backpropagation session, I save both network configurations, the one with the best performance on the validation set and the last training epoch configuration. For performance estimation, I use sensitivity, specificity, positive predictivity, negative predictivity, and accuracy metrics. For validation performance estimation in the case of biased data distribution (for example, in my Face Detection article, you may find that there are a lot more non-faces than faces, 19:1 ratio) I provide a geometric mean and F-measure metrics to support the scenario. To support a large amount of data vectors, I provide File Mapping based data load. That allows to map your hundreds of megs of training data to memory in 1 msec and start your training session immediately. For relatively small amounts of data, you may use a text file format. Finally, the console implementation is easier to use, you avoid a lot of mouse clicking in GUI applications, and may automate the process with batch files for choosing the right network topology, the best performance on the validation and test sets, and so on.)
背景(Background)
看看CodeProject神经网络文章.我还使用了(Have a look at the CodeProject neural network articles. I also used tutorials from the) 第5代(generation5) 现场.对于有偏见的数据分发问题,我使用了:(site. For biased data distribution problems, I used:) 评估分类器的班级分布不均问题(Evaluation of classifiers for an uneven class distribution problem) .在那里查看以了解几何均值和F度量指标.(. Have a look there to understand geometric mean and F-measure metrics.)
使用代码(Using the code)
控制台的帮助行如下所示:(The help line to console is shown below:)
argv[1] t-train
argv[2] network conf file
argv[3] cls1 files [0.9]
argv[4] cls2 files [0.1]
argv[5] epochs num
argv[6] [validation class]
argv[7] [test class]
argv[8] [validation TH 0.5]
argv[9] [vld metric mse]
argv[10] [norm]: [0-no], 1-minmax, 2-zscore, 3-softmax, 4-energy
argv[11] [error tolerance cls] +- 0.05 default
argv[1] r-run
argv[2] network conf file
argv[3] cls files
argv[4] [validation TH 0.5]
argv[5] [norm]: [0-no], 1-minmax, 2-zscore, 3-softmax, 4-energy
ann1dn.exe t net.nn cls1 cls2 3000 [tst.txt][val.txt]
[TH [0.5]][val type [mse]] [norm [0]] [err [0.05]]
ann1dn.exe r net.nn testcls [TH [0.5]] [norm [0]]
metrics: [0 - mse]
1 - AC
2 - sqrt(SE*SP)
3 - sqrt(SE*PP)
4 - sqrt(SE*SP*AC)
5 - sqrt(SE*SP*PP*NP*AC)
6 - F-measure b=1
7 - F-measure b=1.5
8 - F-measure b=3
开始训练的最少参数数量:(The minimal number of parameters to start a training session:)
>ann1dn.exe t network.nn data1_file data2_file 1000
它将使用(It will use the)**network.nn(network.nn)**文件作为神经网络,并加载数据表格(file as a neural network, and load data form)**data1_file(data1_file)**和(and)data2_file(data2_file),代表来自肯定和否定类别的数据向量,并将其训练1000个纪元.(, which represents data vectors from positive and negative classes, and train it for 1000 epochs.)
神经网络文件格式在(The neural network file format is described in my) 人脸检测文章(Face Detection article) .要在训练之前从随机初始化的权重开始,只需在该文件中提供层数和每层神经元数.例如,在演示zip中,您会找到一个(. To start with random initialized weights before the training session, you need to provide only the number of layers and the number of neurons per layer in that file. For example, in the demo zip, you will find a)**虹膜(iris.nn)**文件:(file:)
3
4 8 1
三层,每层有4\8和1个神经元.(Three layers, and 4, 8, and 1 neurons per layer.)
它支持两种数据文件格式.文本一是:(It supports two data file formats. The text one is:)
vector_name class_mark
x1 x2 x3 ... xN
...
vector_name
是您特定向量的名称;它不应以数字元素开头,而只能以字母开头.(is the name of your particular vector; it should not start with a numeric element, only with a letter.) class_mark
是与class属性相对应的非零数字:1\2等.在控制台应用程序中,我仅将1用作正类,并获得0.9的期望输出,将2用作负数,而将0.1作为期望的输出.下一行包含整数或浮点格式的向量条目.(is the non zero number corresponding to the class attribute: 1, 2, etc. In the console application, I use only 1 as positive class with a 0.9 desired output, and 2 as negative with 0.1 as the desired output. The next line contains the vector entries in integer or floating point format.)
在演示zip中,IRIS数据是使用带有四维条目的文本格式组织的:(In the demo zip, the IRIS data is organized using a text format with four-dimensional entries:)
virgi1 2
64 28 56 22
virgi2 2
67 31 56 24
virgi3 2
63 28 51 15
virgi4 2
69 31 51 23
virgi5 2
65 30 52 20
virgi6 2
65 30 55 18
...
setosa1 1
50 33 14 2
setosa2 1
46 34 14 3
setosa3 1
46 36 10 2
setosa4 1
51 33 17 5
setosa5 1
55 35 13 2
...
当您有大量数据时,二进制浮点文件格式很方便.数据以二进制格式以浮点数序列的形式保存在单独的文件中,每个浮点数使用4个字节:(The binary floating point file format is expedient when you have a large amount of data. The data is saved in a separate file as a sequence of floating point numbers in binary format, using 4 bytes per floating point number:)
file1.dat(file1.dat)(2x3矩阵)((2x3 matrix))
[x11] [x12] [x13] [x21] [x22] [x23]
并且,数据矩阵的维以相同的名称保存在文件中,但带有(And, the dimensions of the data matrix are saved in the file with the same name but with a)**.hea(.hea)**延期.(extension.)
file1.hea(file1.hea)
2 3
在上一个示例中,文件(In the previous example, the file)**file1.dat(file1.dat)**包含两个三维向量.(contains two three-dimensional vectors.)
在这种情况下,(In that case, your)**data1_file(data1_file)**要么(or)**data2_file(data2_file)**可能包含具有文件完整路径和类标记的条目:(may contain the entries with the full path to the files and the class marks:)
fullpath\file1.dat 1
fullpath\file2.dat 1
...
控制台应用程序用于反向传播训练的下一个参数是可选的.您可以将它们用于网络的验证和测试,输入数据的标准化以及培训过程中的错误限制.(The next parameters to the console application for backprop training are optional. You may use them for validation and testing of your network, for input data normalization, and error limits during training process.)
>ann1dn t network.nn data1_file data2_file 1000 vld_file tst_file 0.5 2 2
此命令行说明了您在(This command line demonstrates that you use your validation and test sets in the)**vld_file(vld_file)**和(and)**tst_file(tst_file)**文本或二进制格式的文件,如上所述,验证阈值为0.5(即,网络输出大于0.5将数据矢量归为肯定类),敏感度和特异性的几何平均值作为验证停止的性能指标和Zscore规范化.允许的验证指标在控制台帮助行的末尾指定.如果你的(files in text or binary format, as described above, with a validation threshold 0.5 (that is, the network output greater than 0.5 attributes a data vector to a positive class), with geometric mean of sensitivity and specificity as the performance metric for validation stopping and Zscore normalization. The allowed validation metrics are specified at the end of the console help line. If your)**vld_file(vld_file)**要么(or)**tst_file(tst_file)**是空文件,则相应的数据集将由您的训练集中随机选择的条目组成,每个班级都有25%的记录.(are empty files, then the corresponding data set will be composed of randomly selected entries from your training set, with 25% records from each class.)
控制台的最后一个第十一个参数是容错能力.如果正输出和负输出的期望输出与平均网络输出之间的差小于连续10个时期的误差,则训练停止.在反向传播训练期间,如果特定向量的期望输出和网络输出之间的差小于您指定的误差,则不会调整网络权重.这允许为尚未"存储"的数据向量校正网络权重连接.(The last eleventh argument to the console is the error tolerance. If the difference between the desired output and the mean network output for positive and negative classes is less than the error for 10 consecutive epochs, the training stops. During the backpropagation training, if the difference between the desired output and the network output for a particular vector is less than the error you specify, the network weights are not adjusted. This allows to correct the network weight connections for the data vectors which are not yet ‘memorized’.)
下一个示例演示了IRIS数据的样本培训课程:(The next example demonstrates the sample training session for the IRIS data:)
>ann1dn t iris.nn setosa_versi.dat virgi.dat 200 void void 0.5 2 3
loading data...
cls1: 100 cls2: 50 files loaded. size: 4 samples
validaton size: 25 12
validaton size: 26 13
normalizing minmax...
training...
epoch: 1 out: 0.555723 0.478843 max acur: 0.92 (epoch 1) se:84.00 sp:100.00 ac:89.19
epoch: 2 out: 0.582674 0.400396 max acur: 0.92 (epoch 1) se:84.00 sp:100.00 ac:89.19
epoch: 3 out: 0.626480 0.359573 max acur: 0.92 (epoch 3) se:84.00 sp:100.00 ac:89.19
epoch: 4 out: 0.655483 0.326918 max acur: 0.94 (epoch 4) se:96.00 sp:91.67 ac:94.59
epoch: 5 out: 0.699125 0.323879 max acur: 0.94 (epoch 5) se:88.00 sp:100.00 ac:91.89
epoch: 6 out: 0.715539 0.299085 max acur: 0.94 (epoch 6) se:88.00 sp:100.00 ac:91.89
epoch: 7 out: 0.733927 0.292526 max acur: 0.96 (epoch 7) se:92.00 sp:100.00 ac:94.59
epoch: 8 out: 0.750638 0.278721 max acur: 0.98 (epoch 8) se:96.00 sp:100.00 ac:97.30
epoch: 9 out: 0.774599 0.277550 max acur: 0.98 (epoch 8) se:96.00 sp:100.00 ac:97.30
epoch: 10 out: 0.774196 0.256110 max acur: 0.98 (epoch 8) se:96.00 sp:100.00 ac:97.30
epoch: 11 out: 0.793877 0.260753 max acur: 0.98 (epoch 8) se:96.00 sp:100.00 ac:97.30
epoch: 12 out: 0.806802 0.245758 max acur: 0.98 (epoch 8) se:96.00 sp:100.00 ac:97.30
epoch: 13 out: 0.804381 0.228810 max acur: 0.98 (epoch 13) se:96.00 sp:100.00 ac:97.30
epoch: 14 out: 0.814079 0.218740 max acur: 0.98 (epoch 13) se:96.00 sp:100.00 ac:97.30
epoch: 15 out: 0.827635 0.223827 max acur: 0.98 (epoch 13) se:96.00 sp:100.00 ac:97.30
epoch: 16 out: 0.832102 0.210360 max acur: 0.98 (epoch 13) se:96.00 sp:100.00 ac:97.30
epoch: 17 out: 0.840352 0.213165 max acur: 0.98 (epoch 17) se:96.00 sp:100.00 ac:97.30
epoch: 18 out: 0.848957 0.201766 max acur: 0.98 (epoch 18) se:96.00 sp:100.00 ac:97.30
epoch: 19 out: 0.844319 0.188338 max acur: 0.98 (epoch 19) se:96.00 sp:100.00 ac:97.30
epoch: 20 out: 0.856258 0.184954 max acur: 0.98 (epoch 19) se:96.00 sp:100.00 ac:97.30
epoch: 21 out: 0.853244 0.178349 max acur: 0.98 (epoch 19) se:96.00 sp:100.00 ac:97.30
epoch: 22 out: 0.867145 0.185852 max acur: 0.98 (epoch 22) se:96.00 sp:100.00 ac:97.30
epoch: 23 out: 0.863079 0.171684 max acur: 0.98 (epoch 23) se:96.00 sp:100.00 ac:97.30
epoch: 24 out: 0.870108 0.170253 max acur: 0.98 (epoch 24) se:96.00 sp:100.00 ac:97.30
epoch: 25 out: 0.873538 0.164185 max acur: 0.98 (epoch 25) se:96.00 sp:100.00 ac:97.30
epoch: 26 out: 0.871584 0.150496 max acur: 1.00 (epoch 26) se:100.00 sp:100.00 ac:100.00
epoch: 27 out: 0.879310 0.161155 max acur: 1.00 (epoch 26) se:100.00 sp:100.00 ac:100.00
epoch: 28 out: 0.879986 0.154784 max acur: 1.00 (epoch 26) se:100.00 sp:100.00 ac:100.00
epoch: 29 out: 0.880308 0.139083 max acur: 1.00 (epoch 26) se:100.00 sp:100.00 ac:100.00
epoch: 30 out: 0.890360 0.149518 max acur: 1.00 (epoch 26) se:100.00 sp:100.00 ac:100.00
epoch: 31 out: 0.888561 0.145144 max acur: 1.00 (epoch 26) se:100.00 sp:100.00 ac:100.00
epoch: 32 out: 0.880072 0.129197 max acur: 1.00 (epoch 32) se:100.00 sp:100.00 ac:100.00
epoch: 33 out: 0.896553 0.139937 max acur: 1.00 (epoch 32) se:100.00 sp:100.00 ac:100.00
epoch: 34 out: 0.893467 0.137607 max acur: 1.00 (epoch 32) se:100.00 sp:100.00 ac:100.00
epoch: 35 out: 0.893400 0.125793 max acur: 1.00 (epoch 32) se:100.00 sp:100.00 ac:100.00
epoch: 36 out: 0.905036 0.139306 max acur: 1.00 (epoch 32) se:100.00 sp:100.00 ac:100.00
epoch: 37 out: 0.900872 0.118167 max acur: 1.00 (epoch 32) se:100.00 sp:100.00 ac:100.00
epoch: 38 out: 0.909384 0.134014 max acur: 1.00 (epoch 32) se:100.00 sp:100.00 ac:100.00
training done.
training time: 00:00:00:031
classification results: maxacur.nn
train set: 49 25
sensitivity: 100.00
specificity: 100.00
+predictive: 100.00
-predictive: 100.00
accuracy: 100.00
validation set: 25 12
sensitivity: 100.00
specificity: 100.00
+predictive: 100.00
-predictive: 100.00
accuracy: 100.00
test set: 26 13
sensitivity: 88.46
specificity: 92.31
+predictive: 95.83
-predictive: 80.00
accuracy: 89.74
classification results: iris.nn
train set: 49 25
sensitivity: 97.96
specificity: 100.00
+predictive: 100.00
-predictive: 96.15
accuracy: 98.65
validation set: 25 12
sensitivity: 96.00
specificity: 100.00
+predictive: 100.00
-predictive: 92.31
accuracy: 97.30
test set: 26 13
sensitivity: 88.46
specificity: 100.00
+predictive: 100.00
-predictive: 81.25
accuracy: 92.31
网络配置保存到(The network configuration is saved to)**maxacur.nn(maxacur.nn)**对应于验证集上的最佳性能,最后一个纪元配置保存到(corresponding to the best performance on the validation set, and the last epoch configuration is saved to)虹膜(iris.nn).最后,您可以比较它们的结果.(. At the end, you may compare the results for them.)
要使用训练有素的网络进行测试,只需使用以下参数运行控制台:(To use your trained networks for testing, just run the console with these parameters:)
>ann1dn r iris.nn test_data
的(The)**测试数据(test_data)**文件为文本或二进制格式.您可以提供阈值作为第四个参数(默认值为0.5),以获取测试集上的ROC曲线,例如,在0.0到1.0之间变化.(file is in text or binary format. You may provide as the fourth argument the threshold (default is 0.5) to obtain the ROC curve on your test set, for example, varying it between 0.0 and 1.0.)
神经网络类(Neural Network Classes)
神经网络由以下几类组成:(The neural network is composed from the following classes:)
ANNetwork
ANNLayer
ANeuron
ANLink
的(The)ANNetwork
该类包含针对库用户的神经网络的实现.为了避免对其余类使用受保护的接口编程,我使用了(class contains the implementation of the neural network for users of the library. To avoid protected interface programming for the rest of the classes, I used)friend
s.我将首先描述库的结构,然后从中提供您需要使用的功能.(s. I’ll describe the library structure first, and then provide the functions you need to use from the)ANNetwork
类以维护您自己的实现.(class to maintain your own implementations.)
的(The) ANNetwork
包含一组(contains an array of) ANNLayer
层.每层包含一个数组(layers. Every layer contains an array of) ANeuron
神经元对象,每个神经元都包含(neuron objects, and every neuron contains arrays of) ANLink
输入和输出连接.通过这种设计,您可以安排任何所需的网络结构.但是,在我的实现中,我仅提供前馈完整的连接主义结构.(input and output connections. With that design, you may arrange any desired network structure; however, in my implementation, I provide only feed-forward full connectionist structure.)
神经网络的基本单位是神经元类,(The basic unit of the neural network is the neuron class,) ANeuron
.您可以向其添加偏置或输入连接,表示为(. You may add bias or input connection to it, represented as the) ANLink
目的.(object.)
void ANeuron::add_bias()
void ANeuron::add_input(ANeuron *poutn)
众所周知,偏置连接始终将1.0f作为输入值.用(The bias connection always take 1.0f as an input value, as you know. With)add_input()
,您可以添加到神经元的连接,并为其参数提供来自其连接的上一层的神经元.(, you add connection to the neuron, supplying with its argument the neuron from the previous layer to which it connects.)
void ANeuron::add_input(ANeuron *poutn)
{
//poutn - Neuron from previous layer
ANLink *plnk = new ANLink(this, poutn);
inputs.push_back(plnk);
if (poutn)
poutn->outputs.push_back(plnk);
}
因此,每个神经元都"知道"来自下一层的哪些神经元连接到其输出.的(So, every neuron ‘knows’ which neurons from the next layer connect to its output. The) ANLink
就像"箭头",指向上一层的神经元,(is like the ‘arrow’, pointing from the neuron in the previous layer,) ANLink::poutput_neuron
进入下一层的神经元(to the neuron in the next layer,) ANLink::pinput_neuron
.(.)
我以这种方式组织了一个完整的连接主义神经网络结构:(I organize a full connectionist neural network structure in this way:)
void ANNetwork::init_links(const float *avec, const float *mvec, int ifunc, int hfunc)
{
ANNLayer *plr; //current layer
ANNLayer *pprevlr; //previous layer
ANeuron *pnrn; //neuron pointer
int l = 0;
/////////////////////////input layer/////////////////////////////
plr = layers[l++];
swprintf(plr->layer_name, L"input layer");
for (int n = 0; n < plr->get_neurons_number(); n++) {
pnrn = plr->neurons[n];
pnrn->function = ifunc;
pnrn->add_input();
//one input link for every "input layer" neuron
if (avec)
pnrn->inputs[0]->iadd = avec[n];
if (mvec)
pnrn->inputs[0]->w = mvec[n];
else
pnrn->inputs[0]->w = 1.0f;
}
/////////////////////////////////////////////////////////////////
////////////////////////hidden layer's/////////////////////////////////////// 1bias
for (int i = 0; i < m_layers_number - 2 ; i++) { //1input [l-2 hidden] 1output
pprevlr = plr;
plr = layers[l++];
swprintf(plr->layer_name, L"hidden layer %d", i + 1);
for (int n = 0; n < plr->get_neurons_number(); n++) {
pnrn = plr->neurons[n];
pnrn->function = hfunc;
pnrn->add_bias();
for (int m = 0; m < pprevlr->get_neurons_number(); m++)
pnrn->add_input(pprevlr->neurons[m]);
}
}
//////////////////////////////////////////////////////////////////////////////
////////////////////////output layer///////////////////////////////////////// 1bias
pprevlr = plr;
plr = layers[l++];
swprintf(plr->layer_name, L"output layer");
for (int n = 0; n < plr->get_neurons_number(); n++) {
pnrn = plr->neurons[n];
pnrn->function = hfunc;
pnrn->add_bias();
for (int m = 0; m < pprevlr->get_neurons_number(); m++)
pnrn->add_input(pprevlr->neurons[m]);
}
//////////////////////////////////////////////////////////////////////////////
}
的(The) ANeuron
导致神经元"发射"(即从其输入获取数据并将其处理为输出)的函数是:(functions that induce the neuron to ‘fire’, that is to take data from its inputs and process them to its outputs, are:)
void ANeuron::input_fire()
void ANeuron::fire()
第一个仅用于输入层神经元.的(The first one is used for input layer neurons only. The)ANLink
包含一个附加术语,(contains an additional term,)iadd
,用于规范化.最后一个用于隐藏和输出层神经元:(, used in normalization. The last one is used for hidden and output layer neurons:)
inline void ANeuron::input_fire()
{
//input layer normalization
oval = (inputs[0]->ival + inputs[0]->iadd) * inputs[0]->w;
//single input for input layer neuron
switch (function) {
default:
case LINEAR:
break;
case SIGMOID:
oval = 1.0f / (1.0f + exp(float((-1.0f) * oval)));
break;
}
//transfer my output to links connected to my output
for (int i = 0; i < get_output_links_number(); i++)
outputs[i]->ival = oval;
}
inline void ANeuron::fire()
{
//oval = SUM (in[]*w[])
oval = 0.0f;
//compute output for Neuron
for (int i = 0; i < get_input_links_number(); i++)
oval += inputs[i]->ival * inputs[i]->w;
switch (function) {
default:
case LINEAR:
break;
case SIGMOID:
oval = 1.0f / (1.0f + exp(float((-1.0f) * oval)));
break;
}
//transfer my output to links connected to my output
for (int i = 0; i < get_output_links_number(); i++)
outputs[i]->ival = oval;
}
现在,您对库的内部结构有了一些了解,此外,我将描述(Now, you have some idea of the library internals, and further, I’ll describe the) ANNetwork
您可以用于自己的实现的类.(class which you can use for your own implementations.)
您可以从文件加载神经网络,或安排其结构以指定层数和每层神经元:(You may load the neural network from the file, or arrange its structure specifying the number of layers and neurons per layer:)
ANNetwork::ANNetwork(const wchar_t *fname);
ANNetwork::ANNetwork(int layers_number, int *neurons_per_layer);
int nerons_per_layer[4] = {128, 64, 32, 10};
ANNetwork *ann = new ANNetwork(4, neurons_per_layer);
ann->init_links(); //feed-forward full connectionist structure
ann->randomize_weights();
如果要使用循环或任何其他连接来定制神经网络配置,则必须提供自己的功能.(If you want your custom neural network configuration with recurrent or any other connections, you have to provide your own functions.)
的(The) ANNetwork::status()
函数在构造后返回类状态.负值表示错误,0-成功从文件加载网络,以及1-随机权重.(function returns the class status after construction. Negative values indicate error, 0 - network loaded from file with success, and 1 - random weights.)
为了训练,分类和保存网络,提供了以下功能:(To train, classify, and save your network, the following functions are provided:)
bool ANNetwork::train(const float *ivec, float *ovec, const float *dsrdvec, float error = 0.05);
void ANNetwork::classify(const float *ivec, float *ovec);
bool ANNetwork::save(const wchar_t *fname) const;
ivec
和(and)ivec
代表输入到神经网络的输入向量和代表其存储结果的输出向量.它们的尺寸应匹配网络结构中输入和输出神经元的数量.(represent the input vector fed to a neural network and the output vector where it stores its results. Their dimensions should match the number of input and output neurons in the network structure.)dsrdvec
是所需的输出向量,它可以调整其连接以使其与(is the desired output vector to which it adjusts its connections to match it within the)error
公差.的(tolerance. The)ANNetwork::train()
如果发生反向传播,函数将返回true;如果网络输出在内部,则返回false(function will return true in case the backpropogation took place, or false if the network output was within)error
到所需的向量.(to the desired vector.)
反向传播功能使用以下代码:(The backpropagation function uses this code:)
bool ANNetwork::train(const float *ivec, float *ovec, const float *dsrdvec, float error)
// 0.0 - 1.0 learning
{
float dst = 0.0f;
//run network, computation of inputs to output
classify(ivec, ovec);
for (int n = 0; n < layers[m_layers_number-1]->get_neurons_number(); n++) {
dst = fabs(ovec[n] - dsrdvec[n]);
if (dst > error) break;
}
if (dst > error) {
backprop_run(dsrdvec); //it was trained
return true;
} else //it wasnt trained
return false;
}
void ANNetwork::backprop_run(const float *dsrdvec)
{
float nrule = m_nrule; //learning rule
float alpha = m_alpha; //momentum
float delta, dw, oval;
//get deltas for "output layer"
for (int n = 0; n < layers[m_layers_number-1]->get_neurons_number(); n++) {
oval = layers[m_layers_number-1]->neurons[n]->oval;
layers[m_layers_number-1]->neurons[n]->delta =
oval * (1.0f - oval) * (dsrdvec[n] - oval);
}
//get deltas for hidden layers
for (int l = m_layers_number - 2; l > 0; l--) {
for (int n = 0; n < layers[l]->get_neurons_number(); n++) {
delta = 0.0f;
for (int i = 0; i < layers[l]->neurons[n]->get_output_links_number(); i++)
delta += layers[l]->neurons[n]->outputs[i]->w *
layers[l]->neurons[n]->outputs[i]->pinput_neuron->delta;
oval = layers[l]->neurons[n]->oval;
layers[l]->neurons[n]->delta = oval * (1 - oval) * delta;
}
}
////////correct weights for every layer///////////////////////////
for (int l = 1; l < m_layers_number; l++) {
for (int n = 0; n < layers[l]->get_neurons_number(); n++) {
for (int i = 0; i < layers[l]->neurons[n]->get_input_links_number(); i++) {
//dw = rule*Xin*delta + moment*dWprv
dw = nrule * layers[l]->neurons[n]->inputs[i]->ival *
layers[l]->neurons[n]->delta;
dw += alpha * layers[l]->neurons[n]->inputs[i]->dwprv;
layers[l]->neurons[n]->inputs[i]->dwprv = dw;
//correct weight
layers[l]->neurons[n]->inputs[i]->w += dw;
}
}
}
}
现在,您可以组成自己的网络,并进行OCR,计算机视觉等中使用的典型分类任务.(Now, you may compose your own networks and proceed to typical classification tasks used in OCR, computer vision, and so on.)
许可
本文以及所有相关的源代码和文件均已获得The Code Project Open License (CPOL)的许可。
VC7.0 VC7.1 VC8.0 C++ VC6 Windows Dev 新闻 翻译