gpunn.aspx.htm

文章介绍如何使用CUDA实现神经网络

neuron value is the most possible digit. We shade candidates with
different depth of red colors according to their possibilities. </p>

<p>On the right, the user interface will print out feature maps of the first three layers.</p>

<p>Note that C# under Windows XP has a resolution issue. We tested our
program under 120dpi. A 96dpi resolution setting could shift the input
image around, so that the accuracy is badly affected. </p>
No training part is included in our GPU implementation. We use Mike鈥檚 code to train all the weights and cached them with files. 
<h2>Result </h2>

<h4>Accuracy </h4>

<p>Our Neural Network can achieve a 95% accuracy. The database we used
to train the network is called MNIST containing 60000 handwriting
examples from different people. It is reported by Dr. LeCun that this
network can converge after around 25 times of training. This number is
confirmed by our test. We achieved only around 1400 miss-recognition
samples out of 60000 inputs.</p>

<p>Also note that there is a bug in Mike's code. This is the corrected code for initializing the second layer: </p>

<pre lang="C++"><span class="code-keyword">for</span> ( fm=0; fm<span class="code-keyword">&lt;</span><span class="code-digit">50</span>; ++fm)
{
  <span class="code-keyword">for</span> ( ii=0; ii<span class="code-keyword">&lt;</span><span class="code-digit">5</span>; ++ii )
  {
    <span class="code-keyword">for</span> ( jj=0; jj<span class="code-keyword">&lt;</span><span class="code-digit">5</span>; ++jj )
    {
      <span class="code-comment">//</span><span class="code-comment"> iNumWeight = fm * 26;  // 26 is the number of weights per feature map</span>
      iNumWeight = fm * <span class="code-digit">156</span>;  <span class="code-comment">//</span><span class="code-comment"> 156 is the number of weights per feature map</span>
      NNNeuron&amp; n = *( pLayer-<span class="code-keyword">&gt;</span>m_Neurons[ jj + ii*<span class="code-digit">5</span> + fm*<span class="code-digit">25</span> ] );

      n.AddConnection( ULONG_MAX, iNumWeight++ );  <span class="code-comment">//</span><span class="code-comment"> bias weight</span>

      <span class="code-keyword">for</span> ( kk=0; kk<span class="code-keyword">&lt;</span><span class="code-digit">25</span>; ++kk )
      {
        <span class="code-comment">//</span><span class="code-comment"> note: max val of index == 1013, corresponding to 1014 neurons in prev layer</span>
        n.AddConnection(       <span class="code-digit">2</span>*jj + <span class="code-digit">26</span>*ii + kernelTemplate2[kk], iNumWeight++ );
        n.AddConnection( <span class="code-digit">169</span> + <span class="code-digit">2</span>*jj + <span class="code-digit">26</span>*ii + kernelTemplate2[kk], iNumWeight++ );
        n.AddConnection( <span class="code-digit">338</span> + <span class="code-digit">2</span>*jj + <span class="code-digit">26</span>*ii + kernelTemplate2[kk], iNumWeight++ );
        n.AddConnection( <span class="code-digit">507</span> + <span class="code-digit">2</span>*jj + <span class="code-digit">26</span>*ii + kernelTemplate2[kk], iNumWeight++ );
        n.AddConnection( <span class="code-digit">676</span> + <span class="code-digit">2</span>*jj + <span class="code-digit">26</span>*ii + kernelTemplate2[kk], iNumWeight++ );
        n.AddConnection( <span class="code-digit">845</span> + <span class="code-digit">2</span>*jj + <span class="code-digit">26</span>*ii + kernelTemplate2[kk], iNumWeight++ );
      }
    }
  }
} </pre>

<p>Please refer to <a href="http://www.codeproject.com/KB/library/NeuralNetRecognition.aspx?fid=364895&amp;select=2003444&amp;fr=1#xx0xx">this</a> for the details about this bug.</p>

<p>Our GPU implementation is based on the correct version, however there isn't too much difference in terms of accuracy.</p>

<h4>Performance </h4>

<p><img alt="performance.PNG" src="GPUNN.aspx_files/performance.PNG" width="372" border="0" height="345" hspace="0"> </p>

<p>The major reason for using GPU to compute Neural Network is to
achieve robustness. The outcome is promising compared to CPU
implementation. As shown in the table above, the executing time of GPU
version, EmuRelease version and CPU version running on one single input
sample is compared. The GPU version speeds up by 270 times compared to
CPU version and 516.6 times compared to EmuRelease version. To be more
accurate, we also considered the IO time consumption of the GPU
version. As we can see, even when the IO time is considered, our method
is 10 times faster. And in practical use, weight values need only be
loaded into the device once. </p>

<h2>History</h2>

<ul>
<li>14<sup>th</sup> March, 2008: Initial post</li>
</ul>



<!-- Article Ends -->


			<!-- Main Page Contents End -->
			
			</div>
			</span>
			
			<form name="aspnetForm" method="post" action="GPUNN.aspx" id="aspnetForm" style="margin: 0pt; padding: 0pt;">
		<div>
		<input name="__VIEWSTATE" id="__VIEWSTATE" value="/wEPDwULLTEwMDUyNjYzMjhkZI6K1umrUq5YcMpnth7Jq5pt0qpj" type="hidden">
		</div>
		

			
			<h2>License</h2>
			<div id="ctl00_LicenseTerms"><p>This article, along with any associated source code and files, is licensed under <a href="http://creativecommons.org/licenses/by-sa/2.5/">The Creative Commons Attribution-ShareAlike 2.5 License</a></p></div>
			
			<h2>About the Authors</h2>
			
					
            
<table width="100%" border="0" cellpadding="0" cellspacing="5">
<tbody><tr valign="top">
<td id="ctl00_AboutAuthorRptr_ctl00_AboutAuthor_memberPhotoTable" style="width: 155px;" valign="top">
	<b><a id="ctl00_AboutAuthorRptr_ctl00_AboutAuthor_memberProfileLink" href="http://www.codeproject.com/script/Membership/Profiles.aspx?mid=1850775">billconan</a></b><br><br>
	<center><img id="ctl00_AboutAuthorRptr_ctl00_AboutAuthor_memberPhoto" src="GPUNN.aspx_files/2f0acf69-c62a-48ae-be46-dcf7cbec1142.gif" style="border-width: 0px;"></center><br>
	<span id="ctl00_AboutAuthorRptr_ctl00_AboutAuthor_memberType" class="SmallText"></span>
		
</td>
		
<td>
	Ph.D. Student<br>University of California, Davis<br>	

	<table>
	<tbody><tr id="ctl00_AboutAuthorRptr_ctl00_AboutAuthor_jobTitleRow">
			<td class="SmallText" nowrap="nowrap">Occupation: </td>
			<td width="100%"><span id="ctl00_AboutAuthorRptr_ctl00_AboutAuthor_memberJobTitle" class="SmallText">Other</span></td>
		</tr>
		

	

	<tr id="ctl00_AboutAuthorRptr_ctl00_AboutAuthor_locationRow">
			<td class="SmallText">Location: </td>
			<td width="100%"><span id="ctl00_AboutAuthorRptr_ctl00_AboutAuthor_memberLocation" class="SmallText"><img src="GPUNN.aspx_files/US.gif" alt="United States" width="16" height="11"> United States</span></td>
		</tr>
		
	</tbody></table>
	
</td>
</tr>
</tbody></table>

					<br>
				
					
            
<table width="100%" border="0" cellpadding="0" cellspacing="5">
<tbody><tr valign="top">
<td id="ctl00_AboutAuthorRptr_ctl01_AboutAuthor_memberPhotoTable" style="width: 155px;" valign="top">
	<b><a id="ctl00_AboutAuthorRptr_ctl01_AboutAuthor_memberProfileLink" href="http://www.codeproject.com/script/Membership/Profiles.aspx?mid=4907063">kavinguy</a></b><br><br>
	<center><img id="ctl00_AboutAuthorRptr_ctl01_AboutAuthor_memberPhoto" src="GPUNN.aspx_files/94c1a661-bd65-4430-8c3c-bcaf79e1e54f.bmp" style="border-width: 0px;"></center><br>
	<span id="ctl00_AboutAuthorRptr_ctl01_AboutAuthor_memberType" class="SmallText"></span>
		
</td>
		
<td>
	Ph.D. student,<br>University of California, Davis<br>	

	<table>
	<tbody><tr id="ctl00_AboutAuthorRptr_ctl01_AboutAuthor_jobTitleRow">
			<td class="SmallText" nowrap="nowrap">Occupation: </td>
			<td width="100%"><span id="ctl00_AboutAuthorRptr_ctl01_AboutAuthor_memberJobTitle" class="SmallText">Other</span></td>
		</tr>
		

	

	<tr id="ctl00_AboutAuthorRptr_ctl01_AboutAuthor_locationRow">
			<td class="SmallText">Location: </td>
			<td width="100%"><span id="ctl00_AboutAuthorRptr_ctl01_AboutAuthor_memberLocation" class="SmallText"><img src="GPUNN.aspx_files/US.gif" alt="United States" width="16" height="11"> United States</span></td>
		</tr>
		
	</tbody></table>
	
</td>
</tr>
</tbody></table>

					<br>
				
			
			<table id="ctl00_PopularArticlesRow" width="100%" border="0" cellpadding="0" cellspacing="0">
			<tbody><tr valign="top">
				<td style="width: 100%;">
					<h2>Other popular General Graphics articles:</h2><ul><li><a href="http://www.codeproject.com/KB/graphics/zedgraph.aspx">A flexible charting library for .NET</a><div class="SmallText">Looking
for a way to draw 2D line graphs with C#? Here's yet another charting
class library with a high degree of configurability, that is also easy
to use.</div></li><li><a href="http://www.codeproject.com/KB/graphics/cximage.aspx">CxImage</a><div class="SmallText">CxImage
is a C++ class to load, save, display, transform BMP, JPEG, GIF, PNG,
TIFF, MNG, ICO, PCX, TGA, WMF, WBMP, JBG, J2K images.</div></li><li><a href="http://www.codeproject.com/KB/graphics/julijanpiechart.aspx">3D Pie Chart</a><div class="SmallText">A class library for drawing 3D pie charts.</div></li><li><a href="http://www.codeproject.com/KB/graphics/specialfx.aspx">Really cool visual FX</a><div class="SmallText">A set of classes for doing stunning visual effects, including water, plasma and fire.</div></li><li><a href="http://www.codeproject.com/KB/graphics/ImageStone.aspx">ImageStone</a><div class="SmallText">An article on a library for image manipulation.</div></li></ul>
					<h2></h2>				
				</td>
				<td>
					<iframe src="GPUNN.aspx_files/ServeHTML_003.htm" scrolling="no" width="300" frameborder="0" height="250"></iframe>
				</td>
			</tr>
		</tbody></table>
		
		
			<div id="ctl00_AddTo_AddTo" style="margin: 10px;">
<script language="JavaScript" type="text/javascript">
var addtoMethod=1;
var AddURL = document.location.href;
var AddTitle = escape(document.title);
DrawLinks(100, 0, "SmallText Bold", "AddTo") 
</script><span class="SmallText Bold">Add this article to: </span> <a class="AddTo" title="Add this page to Digg" onclick="return addto(0);" href="#"><img src="GPUNN.aspx_files/AddTo_Digg.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Del.icio.us" onclick="return addto(1);" href="#"><img src="GPUNN.aspx_files/AddTo_Delicious.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Google" onclick="return addto(2);" href="#"><img src="GPUNN.aspx_files/AddTo_Google.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Windows Live" onclick="return addto(3);" href="#"><img src="GPUNN.aspx_files/AddTo_Live.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Yahoo! MyWeb" onclick="return addto(4);" href="#"><img src="GPUNN.aspx_files/AddTo_Yahoo.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Technorati" onclick="return addto(5);" href="#"><img src="GPUNN.aspx_files/AddTo_Technorati.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Blink" onclick="return addto(6);" href="#"><img src="GPUNN.aspx_files/AddTo_Blink.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Facebook" onclick="return addto(7);" href="#"><img src="GPUNN.aspx_files/AddTo_Facebook.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Furl" onclick="return addto(8);" href="#"><img src="GPUNN.aspx_files/AddTo_Furl.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Simpy" onclick="return addto(9);" href="#"><img src="GPUNN.aspx_files/AddTo_Simpy.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Reddit" onclick="return addto(10);" href="#"><img src="GPUNN.aspx_files/AddTo_Reddit.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Newsvine" onclick="return addto(11);" href="#"><img src="GPUNN.aspx_files/AddTo_Newsvine.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Stumbleupon" onclick="return addto(12);" href="#"><img src="GPUNN.aspx_files/AddTo_stumbleupon.png" width="16" align="absmiddle" border="0" height="16"></a>  <a class="AddTo" title="Add this page to Mr. Wong" onclick="return addto(13);" href="#"><img src="GPUNN.aspx_files/AddTo_MrWong.png" width="16" align="absmiddle" border="0" height="16"></a>  
</div>
			
			<table id="ctl00_RateArticleRow" class="RatingBar" width="100%" border="0" cellpadding="0" cellspacing="0">
			<tbody><tr>
				<td><a href="#_top">Article Top</a></td>
				<td align="right">            
<table width="100%" cellpadding="0" cellspacing="0"><tbody><tr>

	<td id="ctl00_RateArticle_VoteResultDiv" width="100%" align="right