unh_cmac.ps

一个老外写的CMAC Neural Network源代码和说明

[66.586 0 0 -66.586 0 0]/Helvetica MF
(1)2034 4342 MS
(1)2195 4630 MS
[91.559 0 0 -91.559 0 0]/Helvetica MF
( )2637 4389 MS ( )2663 4389 MS ( )2688 4389 MS ( )2714 4389 MS ( )2739 4389 MS ( )2765 4389 MS ( )2790 4389 MS ( )2816 4389 MS ( )2841 4389 MS ( )2867 4389 MS
( )2892 4389 MS ( )2918 4389 MS ( )2944 4389 MS ( )2969 4389 MS ( )2995 4389 MS ( )3020 4389 MS (\()3046 4389 MS (1)3076 4389 MS (3)3127 4389 MS (\))3178 4389 MS
[92 0 0 -92 0 0]/_Helvetica MF
(In equation 13, f\()496 4782 MS
[92 0 0 -92 0 0]/Symbol MF
(d)1190 4782 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)1235 4795 MS
[92 0 0 -92 0 0]/_Helvetica MF
(\) represents the one-dimensional primitive which forms the basis of the)1248 4782 MS
(receptive field )496 4902 MS
(sensitivity function)1089 4902 MS
(. In practice, f\()1834 4902 MS
[92 0 0 -92 0 0]/Symbol MF
(d)2421 4902 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)2466 4915 MS
[92 0 0 -92 0 0]/_Helvetica MF
(\) = )2479 4902 MS
[92 0 0 -92 0 0]/Symbol MF
(d)2616 4902 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)2661 4915 MS
[92 0 0 -92 0 0]/_Helvetica MF
( is a simple and effective choice for the)2674 4902 MS
n
744 6 1089 4912 B
f
(sensitivity function, producing piece-wise planar approximations.)496 5015 MS
(Finally, equation 6 which describes the weight adjustment during training must be replaced)646 5171 MS
(by:)496 5277 MS
n
1881 5689 M
1932 5689 L
CM 0.258 0.258 scale
s
SM
n
2140 5689 M
2192 5689 L
CM 0.258 0.258 scale
s
SM
n
2609 5640 M
3064 5640 L
CM 0.258 0.258 scale
s
SM
[91.613 0 0 -91.613 0 0]/Symbol MF
(D)1237 5672 MS
[91.613 0 0 -91.613 0 0]/Helvetica MF
(W)1293 5672 MS
(y)1734 5672 MS
(d)1787 5711 MS
(S)1878 5672 MS
(y)2059 5672 MS
(S)2138 5672 MS
(f)2353 5672 MS
(i)2466 5711 MS
(f)2807 5523 MS
(f)2783 5818 MS
[66.625 0 0 -66.625 0 0]/Helvetica MF
(i)1379 5697 MS
(n)2922 5548 MS
(n)2673 5625 MS
(C)2700 5412 MS
(n)2954 5843 MS
(n)2648 5920 MS
(C)2675 5708 MS
[91.613 0 0 -91.613 0 0]/Symbol MF
(=)1436 5672 MS
(-)1985 5672 MS
[66.625 0 0 -66.625 0 0]/Symbol MF
(=)2710 5625 MS
(=)2686 5920 MS
[149.914 0 0 -149.914 0 0]/Symbol MF
(\345)2674 5533 MS
(\345)2650 5829 MS
[91.613 0 28.313 -91.613 0 0]/Symbol MF
(b)1528 5672 MS
(d)2411 5672 MS
(d)2866 5523 MS
(d)2898 5818 MS
[91.613 0 0 -91.613 0 0]/Helvetica MF
(*)1634 5672 MS
(\()1698 5672 MS
(\()1844 5672 MS
(\))1937 5672 MS
(\()2104 5672 MS
(\))2197 5672 MS (\))2227 5672 MS
(*)2287 5672 MS
(\()2384 5672 MS
(\))2498 5672 MS
(*)2558 5672 MS
(\()2838 5523 MS
(\))2973 5523 MS
(\()2870 5818 MS
(\))3005 5818 MS
[66.625 0 0 -66.625 0 0]/Helvetica MF
(1)2745 5625 MS
(2)2822 5774 MS
(1)2721 5920 MS
[91.613 0 0 -91.613 0 0]/Helvetica MF
( )2617 5818 MS
( )3030 5818 MS
( )3083 5672 MS ( )3108 5672 MS ( )3134 5672 MS ( )3160 5672 MS ( )3185 5672 MS ( )3211 5672 MS ( )3236 5672 MS ( )3262 5672 MS ( )3287 5672 MS ( )3313 5672 MS
( )3338 5672 MS ( )3364 5672 MS ( )3389 5672 MS ( )3415 5672 MS ( )3440 5672 MS ( )3466 5672 MS (\()3491 5672 MS (1)3521 5672 MS (4)3572 5672 MS (\))3624 5672 MS
showpage
%%Page: 7 7
12.52 782.039 translate 72 600 div dup neg scale
0 0 transform .25 add round .25 sub exch .25 add round .25 sub exch itransform translate
[92 0 0 -92 0 0]/Helvetica MF
(7)2495 6197 MS
(Note that equation 6 described a weight adjustment )496 610 MS
[92 0 0 -92 0 0]/Symbol MF
(D)2621 610 MS
[92 0 0 -92 0 0]/Helvetica MF
(W which was added equally to each of)2677 610 MS
(the C adjustable weights representing the C excited receptive fields, while in equation 14 the)496 723 MS
(weight adjustment )496 836 MS
[92 0 0 -92 0 0]/Symbol MF
(D)1264 836 MS
[92 0 0 -92 0 0]/Helvetica MF
(W)1320 836 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)1411 874 MS
[92 0 0 -92 0 0]/Helvetica MF
( for each receptive field is scaled by the magnitude of the receptive field)1424 836 MS
(sensitivity function f\()496 974 MS
[92 0 0 -92 0 0]/Symbol MF
(d)1325 974 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)1370 987 MS
[92 0 0 -92 0 0]/Helvetica MF
(\). If f\()1383 974 MS
[92 0 0 -92 0 0]/Symbol MF
(d)1603 974 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)1648 987 MS
[92 0 0 -92 0 0]/Helvetica MF
(\)  = 1 for all )1661 974 MS
[92 0 0 -92 0 0]/Symbol MF
(d)2153 974 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)2198 987 MS
[92 0 0 -92 0 0]/Helvetica MF
(, equations 13 and 14 reduce to equations 5 and 6.)2211 974 MS
(It is obvious from comparing equations 5 and 6 to equations 13 and 14 that the)646 1137 MS
(implementation of non-constant receptive field sensitivity functions requires an increase in)496 1243 MS
(computational effort. Thus, in many cases the simpler case of f\()496 1356 MS
[92 0 0 -92 0 0]/Symbol MF
(d)3087 1356 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)3132 1369 MS
[92 0 0 -92 0 0]/Helvetica MF
(\)  = 1 is preferable, even)3145 1356 MS
(though it results in piece-wise constant CMAC outputs. When a continuous CMAC output is)496 1469 MS
(important, we generally use f\()496 1582 MS
[92 0 0 -92 0 0]/Symbol MF
(d)1704 1582 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)1749 1595 MS
[92 0 0 -92 0 0]/Helvetica MF
(\) = )1762 1582 MS
[92 0 0 -92 0 0]/Symbol MF
(d)1899 1582 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)1944 1595 MS
[92 0 0 -92 0 0]/Helvetica MF
( , which results in piece-wise planar CMAC outputs. We)1957 1582 MS
(have also experimented with cubic )496 1702 MS
(spline and truncated )1924 1702 MS
(gaussian functions for f\()2778 1702 MS
[92 0 0 -92 0 0]/Symbol MF
(d)3760 1702 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)3805 1715 MS
[92 0 0 -92 0 0]/Helvetica MF
(\).)3818 1702 MS
(Paradoxically, learning system performance is usually worse when using these higher order)496 1815 MS
(sensitivity functions for input dimensions greater than two. We feel that this results from their)496 1921 MS
(much smaller magnitude near the edges of the receptive field, which dominates the receptive)496 2027 MS
(field volume in higher dimensional spaces. In applications we thus use either constant or linear)496 2133 MS
(sensitivity functions. Note that when using non-constant sensitivity functions, the arrangement)496 2239 MS
(of receptive fields is critical to good performance. In such cases, near uniform arrangements of)496 2345 MS
(receptive fields always provide substantially better learning system performance [12] than the)496 2451 MS
(diagonalized arrangement used by )496 2557 MS
(Albus.)1932 2557 MS
[92 0 0 -92 0 0]/Helvetica-Bold MF
(Receptive Field Hashing Considerations)496 2766 MS
[92 0 0 -92 0 0]/Helvetica MF
(As discussed previously, the total number of receptive fields required to span a )646 2923 MS
(multi-)3880 2923 MS
(dimensional space \(C times\) is often too large for practical implementation in terms of the)496 3029 MS
(storage required for the adjustable weights of all possible receptive fields. On the other hand, it)496 3135 MS
(is unlikely that the entire input state space of a large system would be visited in solving a)496 3241 MS
(specific problem. Thus it is only necessary to store information for receptive fields that are)496 3347 MS
(excited during training. Following this logic, most implementations of CMAC neural networks)496 3453 MS
(include some form of pseudo-random hashing to transform the vector virtual address )496 3559 MS
(A)3970 3559 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)4031 3572 MS
[92 0 0 -92 0 0]/Helvetica MF
( of an)4044 3559 MS
n
60 6 3970 3569 B
f
(excited receptive field into a scalar address )496 3665 MS
(A')2280 3665 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)2359 3678 MS
[92 0 0 -92 0 0]/Helvetica MF
( of the corresponding weight storage.)2372 3665 MS
(The major requirement of the hashing function is that the generated addresses )646 3821 MS
(A')3878 3821 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)3957 3834 MS
[92 0 0 -92 0 0]/Helvetica MF
( be)3970 3821 MS
(uniformly distributed in the range M of the available physical memory addresses, even for small)496 3927 MS
(changes in )496 4033 MS
(A)967 4033 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)1028 4046 MS
[92 0 0 -92 0 0]/Helvetica MF
(. In software implementations of CMAC, we have primarily used a simple hashing)1041 4033 MS
n
60 6 967 4043 B
f
(algorithm based on previously generated random number tables:)496 4139 MS
[91.664 0 0 -91.664 0 0]/Symbol MF
(\242)1679 4405 MS
(=)1732 4401 MS
(\346)1810 4298 MS
(\350)1810 4495 MS
(\347)1810 4394 MS
(\347)1810 4430 MS
(\366)2472 4298 MS
(\370)2472 4495 MS
(\367)2472 4394 MS
(\367)2472 4430 MS
[66.664 0 0 -66.664 0 0]/Symbol MF
(=)1897 4497 MS
[150 0 0 -150 0 0]/Symbol MF
(\345)1870 4411 MS
[91.664 0 0 -91.664 0 0]/Helvetica MF
(A)1612 4401 MS
(T)2006 4401 MS
(a)2115 4401 MS
(R)2323 4401 MS
(M)2635 4401 MS ( )2711 4401 MS ( )2737 4401 MS ( )2762 4401 MS ( )2788 4401 MS ( )2813 4401 MS ( )2839 4401 MS ( )2864 4401 MS ( )2890 4401 MS ( )2915 4401 MS
[66.664 0 0 -66.664 0 0]/Helvetica MF
(i)1681 4424 MS
(j)2064 4424 MS
(i)2170 4424 MS (j)2184 4424 MS
(j)2398 4424 MS
(j)1883 4497 MS
(N)1895 4292 MS
[91.664 0 0 -91.664 0 0]/Helvetica MF
([)2088 4401 MS
(%)2220 4401 MS
(])2425 4401 MS
(%)2532 4401 MS
[66.664 0 0 -66.664 0 0]/Helvetica MF
(1)1933 4497 MS
[91.664 0 0 -91.664 0 0]/Helvetica MF
( )2941 4401 MS ( )2966 4401 MS ( )2992 4401 MS ( )3017 4401 MS ( )3043 4401 MS ( )3068 4401 MS ( )3094 4401 MS (\()3119 4401 MS (1)3149 4401 MS (5)3201 4401 MS
(\))3252 4401 MS
[92 0 0 -92 0 0]/Helvetica MF
(where each )496 4670 MS
(T)996 4670 MS
[58 0 0 -58 0 0]/Helvetica MF
(j)1053 4683 MS
[92 0 0 -92 0 0]/Helvetica MF
( represents a table of uniformly distributed random values with )1066 4670 MS
(R)3624 4670 MS
[58 0 0 -58 0 0]/Helvetica MF
(j)3690 4683 MS
[92 0 0 -92 0 0]/Helvetica MF
( total table)3703 4670 MS
(entries. Here, )496 4776 MS
(R)1075 4776 MS
[58 0 0 -58 0 0]/Helvetica MF
(j)1141 4789 MS
[92 0 0 -92 0 0]/Helvetica MF
( effectively limits the dynamic range of the )1154 4776 MS
(jth component of the normalized)2891 4776 MS
(input vector, due to wrap-around of the table index. This hashing algorithm is numerically)496 4882 MS
(efficient and produces a good approximation to uniformly distributed addresses in the range 0)496 4988 MS
(to M-1, as long as the dynamic range of the pseudo-random numbers in the tables is at least M.)496 5094 MS
(Note that a single bit change in any component of )496 5200 MS
(A)2545 5200 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)2606 5213 MS
[92 0 0 -92 0 0]/Helvetica MF
( can cause a large change in )2619 5200 MS
(A')3823 5200 MS
[58 0 0 -58 0 0]/Helvetica MF
(i)3902 5213 MS
[92 0 0 -92 0 0]/Helvetica MF
(, as)3915 5200 MS
n
60 6 2545 5210 B
f
(desired. In our experience, the quality of the hashing produced is sensitive to the quality of the)496 5306 MS
(uniform random number generator used to fill the )496 5412 MS
(T)2522 5412 MS
[58 0 0 -58 0 0]/Helvetica MF
(j)2579 5425 MS
[92 0 0 -92 0 0]/Helvetica MF
( tables, but is not sensitive to the specific)2592 5412 MS
(seed selected when using a good random number generator.)496 5518 MS
(Hashing collisions are defined by:)646 5674 MS
1 j
1 setlinecap
16 sl
n
2347 5844 M
2407 5844 L
CM 0.258 0.258 scale
s
SM
n
2619 5844 M
2680 5844 L
CM 0.258 0.258 scale
s
SM
[91.664 0 0 -91.664 0 0]/Symbol MF
(\242)1638 5832 MS
(=)1739 5827 MS
(\242)1911 5832 MS
(\271)2514 5827 MS
[91.664 0 0 -91.664 0 0]/Helvetica MF
(A)1571 5827 MS
(A)1844 5827 MS
(A)2347 5827 MS
(A)2619 5827 MS
[66.664 0 0 -66.664 0 0]/Helvetica MF
(n)1640 5850 MS
(m)1912 5850 MS
(n)2415 5860 MS
(m)2688 5860 MS
[91.664 0 0 -91.664 0 0]/Helvetica MF
( )1977 5827 MS ( )2002 5827 MS ( )2028 5827 MS ( )2053 5827 MS ( )2079 5827 MS (f)2104 5827 MS (o)2131 5827 MS (r)2182 5827 MS ( )2212 5827 MS ( )2237 5827 MS
( )2263 5827 MS ( )2289 5827 MS ( )2314 5827 MS
( )2752 5827 MS ( )2778 5827 MS ( )2803 5827 MS ( )2829 5827 MS ( )2854 5827 MS ( )2880 5827 MS ( )2905 5827 MS ( )2931 5827 MS ( )2956 5827 MS ( )2982 5827 MS
( )3008 5827 MS ( )3033 5827 MS ( )3059 5827 MS ( )3084 5827 MS ( )3110 5827 MS ( )3135 5827 MS (\()3161 5827 MS (1)3191 5827 MS (6)3242 5827 MS (\))3293 5827 MS
showpage
%%Page: 8 8
12.52 782.039 translate 72 600 div dup neg scale
0 0 transform .25 add round .25 sub exch .25 add round .25 sub exch itransform translate
[92 0 0 -92 0 0]/Helvetica MF
(8)2495 6197 MS
(This has the effect of introducing learning interference, in the sense that training adjustments to)496 603 MS
(two distinct and possibly distant receptive fields affect the same adjustable weights. In many)496 709 MS
(implementations of CMAC \(including