📄 221-225.html
字号:
<HTML>
<HEAD>
<META name=vsisbn content="0849398010">
<META name=vstitle content="Industrial Applications of Genetic Algorithms">
<META name=vsauthor content="Charles Karr; L. Michael Freeman">
<META name=vsimprint content="CRC Press">
<META name=vspublisher content="CRC Press LLC">
<META name=vspubdate content="12/01/98">
<META name=vscategory content="Web and Software Development: Artificial Intelligence: Other">
<TITLE>Industrial Applications of Genetic Algorithms:Tuning Bama Optimized Recurrent Neural Networks Using Genetic Algorithms</TITLE>
<!-- HEADER -->
<STYLE type="text/css">
<!--
A:hover {
color : Red;
}
-->
</STYLE>
<META NAME="ROBOTS" CONTENT="NOINDEX, NOFOLLOW">
<!--ISBN=0849398010//-->
<!--TITLE=Industrial Applications of Genetic Algorithms//-->
<!--AUTHOR=Charles Karr//-->
<!--AUTHOR=L. Michael Freeman//-->
<!--PUBLISHER=CRC Press LLC//-->
<!--IMPRINT=CRC Press//-->
<!--CHAPTER=11//-->
<!--PAGES=221-225//-->
<!--UNASSIGNED1//-->
<!--UNASSIGNED2//-->
<CENTER>
<TABLE BORDER>
<TR>
<TD><A HREF="217-221.html">Previous</A></TD>
<TD><A HREF="../ewtoc.html">Table of Contents</A></TD>
<TD><A HREF="225-231.html">Next</A></TD>
</TR>
</TABLE>
</CENTER>
<P><BR></P>
<P>Based on this performance measure, RNN weights are altered in the direction that is most likely to reduce the RNN error <I>E<SUB>t</SUB>(t)</I>. This alteration is driven by a derivative-based algorithm. The error gradient is derived from Equation (11.6). The error gradient are computed according to</P>
<P ALIGN="CENTER"><IMG SRC="images/11-07d.jpg"></P>
<P>where <I>d(t)</I> = desired (target) output.</P>
<P>Ordered derivatives are a special case of the chain rule only applicable for an ordered system where the values can be computed one by one in the order of <I>E<SUB>1</SUB></I>, <I>E<SUB>2</SUB></I>, … <I>E<SUB>i</SUB></I>. By using the definitions of ordered derivatives presented by Werbos [6], the error derivatives for each neuron are computed according to</P>
<P ALIGN="CENTER"><IMG SRC="images/11-08d.jpg"></P>
<P>where
</P>
<P ALIGN="CENTER"><IMG SRC="images/11-09d.jpg"></P>
<P>The ordered derivatives for the weights can be calculated as
</P>
<P ALIGN="CENTER"><IMG SRC="images/11-10d.jpg"></P>
<P>And, the weights are updated according to
</P>
<P ALIGN="CENTER"><IMG SRC="images/11-11d.jpg"></P>
<P>where ε is a learning rate, which is used for RNN learning speed adjustments. The larger the value of the learning rate, the faster adaptation occurs. However, if the learning rate is too large, the RNN can diverge, leading to increasing errors and erroneous results. On the other hand, the smaller the value of the learning rate, the slower adaptation tends to be, but the more likely the RNN is to converge.
</P>
<P>In RNNs, Equations 11.2 to 11.12 are applied repeatedly until the RNN can produce the desired outputs within a given tolerance. This tolerance is set by the RNN user.</P>
<P>As mentioned earlier in this section, different NNs have very different structures and distinguishing characteristics. Therefore, NN users must select a NN that is most suitable to the particular problem they wish to solve.</P>
<P>BORN is an RNN connection optimization algorithm, which can eliminate unnecessary RNN connections. A brief explanation is provided below. In order to use the BORN algorithm in RNNs with BP learning, a new function <I>g(C<SUB>ij</SUB>)</I> must be introduced into the RNN connections. This function operates on the connections between neurons. If <I>g(C<SUB>ij</SUB>)</I> = <I>1.0</I>, a connection between the <I>i</I><SUP><SMALL>th</SMALL></SUP> and <I>j</I><SUP><SMALL>th</SMALL></SUP> neurons is present. If, however, <I>g(C<SUB>ij</SUB>)</I> = <I>0.0</I>, the connection is not present. In order to apply the BORN algorithm in RNNs, Equation 11.3 must be changed to</P>
<P ALIGN="CENTER"><IMG SRC="images/11-12d.jpg"></P>
<P>where <P ALIGN="CENTER"><IMG SRC="images/11-13d.jpg"></P>
</P>
<P>where α is a constant, typically set to a value greater than 10.0. For sufficiently large α, <I>g(C<SUB>ij</SUB>) ≅ 1.0 for C<SUB>ij</SUB> > 0, and g(C<SUB>ij</SUB>) ≅ 0.0 for C<SUB>ij</SUB> < 0</I>.</P>
<P>If the connection strength, <I>C<SUB>ij</SUB></I>, is set to 0.0 at the beginning of the training, and if BP is applied to update <I>C<SUB>ij</SUB>, g(C<SUB>ij</SUB>)</I> will approach either 1.0 or 0.0. Now, the error derivative can be expressed as</P>
<P ALIGN="CENTER"><IMG SRC="images/11-14d.jpg"></P>
<P>For the RNN, the error gradients for the weights are expressed as
</P>
<P ALIGN="CENTER"><IMG SRC="images/11-15d.jpg"></P>
<P ALIGN="CENTER"><IMG SRC="images/11-16d.jpg"></P>
<P>The weight and connection updates are expressed as
</P>
<P ALIGN="CENTER"><IMG SRC="images/11-17d.jpg"></P>
<P>For RNN training, Equations 11.13 to 11.18 are applied repeatedly until the RNN can produce the desired outputs within some pre-defined tolerance on the RNN error. Further details of the BORN algorithm and example applications can be found in Reference [12].
</P>
<P><FONT SIZE="+1"><B>NN CONNECTIVITIES OPTIMIZATION WITH GA</B></FONT></P>
<P>This section performs BP Neural Network weight selection and connectivity optimizations by GA and compares the results with BORN algorithm results. The purpose of this study is to compare the stability and robustness of the BORN algorithm and GA. Simple GA [2] was used to verify the performance of the Neural Network with GA optimization.
</P>
<P><FONT SIZE="+1"><B><I>Decode and Fitness Function for GA-NN</I></B></FONT></P>
<P>In order to optimize NN connections by GA, a decode routine and a fitness function must be developed. The number of NN connections is defined as the sum of the number of NN inputs, hidden neurons, and outputs. Since, in this chapter, the NN connections are represented in binary form (such as connected is I and disconnected is 0), this can be translated to GA as a 1-bit multi-parameter problem. Therefore, the decode function is applied such as “if an allele is 1, then the NN connection exists; otherwise no connection” for the number of connections, which is essentially the same as the chromosome length. Listing 11.1 is the complete C language listing for decode function used in this study. The objective of the GA is to maximize the fitness function, <B><I>f</I></B>, defined in Equation 11.19.</P>
<P ALIGN="CENTER"><IMG SRC="images/11-18d.jpg"></P>
<!-- CODE //-->
<PRE>
<B>void</B> decode(chromosome *chrom, <B>int</B> *lchrom)
{
<B>int</B> i, j, counter;
counter=0;
<B>for</B>(i=(NN_Num_Input+ NN_Num_Hidden+ NN_Num_Output);i>=1;--i){
<B>for</B>(j=( NN_Num_Input+ NN_Num_Hidden+ NN_Num_Output);j>=1;--j){
<B>if</B>( chrom[counter] ){ ConnectionMatrix[i][j] = 1;} //<I>Connection</I> Exists
<B>else</B> { ConnectionMatrix[i][j] = 0; } //<I>No Connection</I>
++counter;}}
<B>return</B>;
}
</PRE>
<!-- END CODE //-->
<P><B>Listing 11.1</B> Decode function for GA-NN connection optimization problem.</P><P><BR></P>
<CENTER>
<TABLE BORDER>
<TR>
<TD><A HREF="217-221.html">Previous</A></TD>
<TD><A HREF="../ewtoc.html">Table of Contents</A></TD>
<TD><A HREF="225-231.html">Next</A></TD>
</TR>
</TABLE>
</CENTER>
<hr width="90%" size="1" noshade>
<div align="center">
<font face="Verdana,sans-serif" size="1">Copyright © <a href="/reference/crc00001.html">CRC Press LLC</a></font>
</div>
<!-- all of the reference materials (books) have the footer and subfoot reveresed -->
<!-- reference_subfoot = footer -->
<!-- reference_footer = subfoot -->
</BODY>
</HTML>
<!-- END FOOTER -->
⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -