unh_cmac.ps

一个老外写的CMAC Neural Network源代码和说明

[92 0 0 -92 0 0]/Helvetica MF
(In this section we discuss extensions to the conventional )646 3116 MS
(Albus CMAC neural network.)2970 3116 MS
(When appropriate, modifications to the equations of the previous section are presented in a)496 3222 MS
(form intended to facilitate software implementation. When all of these extensions are)496 3328 MS
(implemented, the algorithms and learning system performance can be quite different than for)496 3434 MS
(the conventional )496 3540 MS
(Albus CMAC. However, the extensions are faithful to the original learning)1190 3540 MS
(system concepts of )496 3646 MS
(Albus, and thus are still appropriately called CMAC algorithms. Note that)1311 3646 MS
(some of the frequently described limitations of CMAC \(such as only learning integer mappings\))496 3752 MS
(are in fact characteristics specific to the original )496 3858 MS
(Albus algorithm, and are neither properties of)2454 3858 MS
(nor limitations of the general CMAC concept.)496 3964 MS
[92 0 0 -92 0 0]/Helvetica-Bold MF
(Organization of Receptive Fields)496 4173 MS
[92 0 0 -92 0 0]/Helvetica MF
(The descriptions in the first section apply to the classic )646 4330 MS
(Albus CMAC [2,4-7]. Research at)2893 4330 MS
(UNH [12-14] and elsewhere [14,15] has investigated alternative lattice arrangements for the)496 4436 MS
(receptive fields which provide more uniform local generalization in higher dimensional input)496 4542 MS
(spaces. The receptive field mapping used in the conventional CMAC implementation has three)496 4648 MS
(key features which it is desirable to retain:)496 4754 MS
(1. Each input in the )646 4910 MS
(multi-dimensional input state space falls into exactly the same number)1460 4910 MS
(of receptive fields \(C\), and this number of overlapping fields is not dependent on the)646 5016 MS
(dimensionality N of the space or the total size M of the physical weight memory.)646 5122 MS
(2. Regardless of the operating point in the )646 5278 MS
(multi-dimensional space, a change of one)2384 5278 MS
(quantization level )646 5391 MS
[92 0 0 -92 0 0]/Symbol MF
(D)1380 5391 MS
[58 0 0 -58 0 0]/Helvetica MF
(j)1436 5404 MS
[92 0 0 -92 0 0]/Helvetica MF
( in any input parameter causes exactly one active receptive field to)1449 5391 MS
(become inactive and one new receptive field to become active. This provides for uniform)646 5504 MS
(quantization within the space.)646 5610 MS
(3. The receptive fields are arranged in a geometrically regular way, such that the)646 5766 MS
(coordinates of the C excited receptive fields for any input can easily be determined in)646 5872 MS
showpage
%%Page: 4 4
12.52 782.039 translate 72 600 div dup neg scale
0 0 transform .25 add round .25 sub exch .25 add round .25 sub exch itransform translate
[92 0 0 -92 0 0]/Helvetica MF
(4)2495 6197 MS
(software, or generated in hardware, without having to compare to independent coordinates)646 603 MS
(stored for each of the very many receptive fields.)646 709 MS
(The conventional )646 865 MS
(Albus CMAC implementation achieves these properties by offsetting the)1371 865 MS
(parallel layers of receptive fields along )496 971 MS
(hyperdiagonals in the input space \(the effect of the )2081 971 MS
(s')4173 971 MS
[58 0 0 -58 0 0]/Helvetica MF
(j)4237 984 MS
[92 0 0 -92 0 0]/Helvetica MF
(-i)4250 971 MS
(terms in equation 3\). All inputs fall within the same number of receptive fields. However, some)496 1077 MS
(inputs fall near the centers of several receptive fields while other inputs fall near the centers of)496 1183 MS
(no receptive fields. This results in )496 1289 MS
(inhomogeneous and )1883 1289 MS
(anisotropic generalization within the)2742 1289 MS
(input state space. Ideally, the distribution of receptive fields should be uniform in the )496 1395 MS
(multi-)3936 1395 MS
(dimensional input space unless prior knowledge of the function to be learned dictates)496 1501 MS
(otherwise.)496 1607 MS
(The above three desirable features of the CMAC mapping can be used to place constraints)646 1763 MS
(on the possible arrangements of receptive fields, through which new arrangements can be)496 1869 MS
(generated. For the following discussion, assume that the N-dimensional input space has been)496 1975 MS
(normalized using equation 2 such that the widths of the receptive fields relative to all N)496 2081 MS
(components of the input are equal to C \(the generalization parameter\). The first and third items)496 2187 MS
(above suggest that the arrangement should be periodic in the normalized input space, with)496 2293 MS
(period C. This is equivalent to assuming that the receptive fields will be arranged in C parallel)496 2399 MS
(layers, each with non-overlapping receptive fields, and that only the offsets of each layer)496 2505 MS
(relative to the others can be varied when generating new receptive field distributions. In this)496 2611 MS
(case, the distribution of receptive fields throughout the space is uniquely defined by the)496 2717 MS
(arrangement of C receptive field centers in an N-dimensional )496 2823 MS
(hypercube of side C \(referred to)2999 2823 MS
(as the )496 2929 MS
(reference )773 2929 MS
(hypercube)1189 2929 MS
(\), with one receptive field centered at the corner of the )1617 2929 MS
(hypercube)3846 2929 MS
n
843 6 773 2939 B
f
(\(coordinate <0,0,...0>\).)496 3035 MS
(The individual receptive fields are )646 3191 MS
(hypercubes of side C in the normalized input space, and)2035 3191 MS
(C receptive field centers are located inside of any region bounded by a )496 3297 MS
(hypercube of side C.)3396 3297 MS
(Item 2 in the above list \(uniform segmentation of the space\) is thus only achieved if the C)496 3403 MS
(receptive field centers are spaced uniformly \(with integer separation\) when projected onto each)496 3509 MS
(of the N axes of the reference )496 3615 MS
(hypercube. Any arrangement which satisfies these criteria)1739 3615 MS
(qualifies as a CMAC mapping according to the three items above. However, many of the)496 3721 MS
(possible arrangements \(such as the conventional CMAC mapping\) have locally )496 3827 MS
(nonuniform)3728 3827 MS
(distributions.)496 3933 MS
(Parks and )646 4089 MS
(Militzer [15] studied the arrangement of receptive fields in CMAC networks using)1087 4089 MS
(distance between nearest neighbors as the evaluation criteria, based on the assumption that)496 4195 MS
(the most uniform distribution would have the greatest distance between nearest neighbors)496 4301 MS
(\(given a fixed receptive field density in the space\). They further assumed that the receptive field)496 4407 MS
(centers were arranged in a lattice defined by a )496 4513 MS
(displacement vector)2410 4513 MS
n
816 6 2410 4523 B
f
1 j
1 setlinecap
16 sl
n
1737 4682 M
1792 4682 L
CM 0.258 0.258 scale
s
SM
[91.141 0 0 -91.141 0 0]/Helvetica MF
(D)1729 4666 MS
(d)1987 4666 MS
(d)2124 4666 MS
(d)2421 4666 MS
[66.285 0 0 -66.285 0 0]/Helvetica MF
(N)2470 4689 MS
[91.141 0 0 -91.141 0 0]/Symbol MF
(=)1826 4666 MS
(<)1910 4666 MS
(>)2552 4666 MS
[66.285 0 0 -66.285 0 0]/Helvetica MF
(1)2036 4689 MS
(2)2177 4689 MS
[91.141 0 0 -91.141 0 0]/Helvetica MF
(,)2072 4666 MS
(,)2222 4666 MS
(.)2268 4666 MS
(.)2295 4666 MS
(.)2323 4666 MS
(,)2369 4666 MS
( )2632 4666 MS ( )2658 4666 MS ( )2683 4666 MS ( )2709 4666 MS ( )2734 4666 MS ( )2760 4666 MS ( )2785 4666 MS ( )2811 4666 MS ( )2836 4666 MS ( )2862 4666 MS
( )2887 4666 MS ( )2913 4666 MS ( )2938 4666 MS ( )2964 4666 MS ( )2989 4666 MS ( )3015 4666 MS (\()3040 4666 MS (7)3070 4666 MS (\))3122 4666 MS
[92 0 0 -92 0 0]/Helvetica MF
(such that the coordinate of the )496 4835 MS
(ith receptive field in the reference )1763 4835 MS
(hypercube was)3146 4835 MS
[91.141 0 0 -91.141 0 0]/Symbol MF
(<)902 4988 MS
(>)2658 4988 MS
(=)3006 4988 MS
[91.141 0 0 -91.141 0 0]/Helvetica MF
(\()976 4988 MS
(*)1066 4988 MS
(\))1211 4988 MS
(%)1250 4988 MS
(,)1421 4988 MS
(\()1470 4988 MS
(*)1579 4988 MS
(\))1749 4988 MS
(%)1788 4988 MS
(,)1959 4988 MS
(.)2005 4988 MS
(.)2032 4988 MS
(.)2059 4988 MS
(,)2105 4988 MS
(\()2154 4988 MS
(*)2245 4988 MS
(\))2419 4988 MS
(%)2458 4988 MS
(,)3111 4988 MS
(,)3207 4988 MS
(.)3253 4988 MS
(.)3281 4988 MS
(.)3308 4988 MS
(,)3354 4988 MS
(i)1017 4988 MS
(d)1120 4988 MS
(C)1359 4988 MS
(i)1530 4988 MS
(d)1645 4988 MS
(C)1897 4988 MS
(i)2196 4988 MS
(d)2311 4988 MS
(C)2567 4988 MS
(i)2964 4988 MS
(C)3406 4988 MS ( )3472 4988 MS ( )3497 4988 MS ( )3523 4988 MS ( )3548 4988 MS ( )3574 4988 MS ( )3599 4988 MS ( )3625 4988 MS ( )3650 4988 MS ( )3676 4988 MS
[66.285 0 0 -66.285 0 0]/Helvetica MF
(N)2360 5011 MS
(1)1169 5011 MS
(2)1699 5011 MS
[91.141 0 0 -91.141 0 0]/Helvetica MF
(1)3078 4988 MS
(2)3163 4988 MS
( )3702 4988 MS ( )3727 4988 MS ( )3753 4988 MS ( )3778 4988 MS ( )3804 4988 MS ( )3829 4988 MS ( )3855 4988 MS (\()3880 4988 MS (8)3910 4988 MS (\))3961 4988 MS
[92 0 0 -92 0 0]/Helvetica MF
(In these terms, the conventional CMAC would be defined by a lattice displacement vector of)496 5157 MS
n
1770 5327 M
1825 5327 L
CM 0.258 0.258 scale
s
SM
[91.664 0 0 -91.664 0 0]/Helvetica MF
(D)1762 5310 MS
[66.664 0 0 -66.664 0 0]/Helvetica MF
(A)1837 5343 MS (l)1882 5343 MS (b)1896 5343 MS (u)1934 5343 MS (s)1971 5343 MS
[91.664 0 0 -91.664 0 0]/Symbol MF
(=)2049 5310 MS
(<)2133 5310 MS
(>)2518 5310 MS
[91.664 0 0 -91.664 0 0]/Helvetica MF
(1)2204 5310 MS
(1)2276 5310 MS
(1)2449 5310 MS
(,)2249 5310 MS
(,)2321 5310 MS
(.)2367 5310 MS
(.)2394 5310 MS
(.)2421 5310 MS
( )2598 5310 MS ( )2623 5310 MS ( )2649 5310 MS ( )2674 5310 MS ( )2700 5310 MS ( )2725 5310 MS ( )2751 5310 MS ( )2776 5310 MS ( )2802 5310 MS ( )2827 5310 MS
( )2853 5310 MS ( )2878 5310 MS ( )2904 5310 MS ( )2929 5310 MS ( )2955 5310 MS ( )2981 5310 MS (\()3006 5310 MS (9)3036 5310 MS (\))3087 5310 MS
[92 0 0 -92 0 0]/Helvetica MF
(They performed an exhaustive search of such lattice arrangements for various values of C and)496 5488 MS
(N, and developed tables of best displacement vectors according to their nearest neighbor)496 5594 MS
(distance criteria.)496 5700 MS
(We have developed a simple heuristic for selecting similar displacement vectors for any)646 5856 MS
(values of C and N which provide lattice arrangements equivalent to those found by Parks and)496 5962 MS
showpage
%%Page: 5 5
12.52 782.039 translate 72 600 div dup neg scale
0 0 transform .25 add round .25 sub exch .25 add round .25 sub exch itransform translate
[92 0 0 -92 0 0]/Helvetica MF
(5)2495 6197 MS
(Militzer [14] without performing a search [12-14]. First, we choose the set of integers in the)496 603 MS
(range 1 to C/2 which are not factors of C or integer products of factors of C \(the value 1 can be)496 709 MS
(included in the set\). These are the candidate values for the displacement vector components)496 815 MS
(\(guaranteeing uniform projection of the centers on the axes of the reference )496 921 MS
(hypercube\), from)3611 921 MS
(which N must be selected. If there are more than N candidate values, we choose N \(there are)496 1027 MS
(multiple nearly equivalent arrangements\). If there are less than N candidates, the best that can)496 1133 MS
(be done is to use all of the candidate values with a minimum number of repetitions of any one)496 1239 MS
(value. However, the resulting mapping will be )496 1345 MS
(diagonalized \(locally )2364 1345 MS
(nonuniform\) in some)3214 1345 MS
(projections to lower dimensional spaces. A better solution in this case is to increase C, in order)496 1451 MS
(to achieve at least N candidates for the displacement vector.)496 1557 MS
(For a CMAC with a three-dimensional input and a generalization of 16, the candidate values)646 1713 MS
(for the displacement vector are 1, 3, 5, and 7. A typical displacement vector might be <1,3,5>)496 1819 MS
(which would produce receptive field locations at <0,0,0>, <1,3,5>, <2,6,10>, and so forth, within)496 1925 MS
(the reference cube.)496 2031 MS
(The CMAC computations described in the first section can easily be modified to)646 2187 MS
(accommodate the displacement vector approach to specifying receptive field placement. In this)496 2293 MS
(case, the virtual addresses of the excited receptive fields are given by)496 2399 MS
1 j
1 setlinecap
16 sl
n
773 2570 M
834 2570 L
CM 0.258 0.258 scale
s
SM
[91.438 0 0 -91.438 0 0]/Helvetica MF
(A)773 2553 MS
(s)1066 2553 MS
(s)1306 2553 MS
(i)1483 2553 MS
(d)1573 2553 MS
(C)1773 2553 MS
(s)1947 2553 MS
(s)2199 2553 MS
(i)2388 2553 MS
(d)2478 2553 MS
(C)2691 2553 MS
(s)3012 2553 MS
(s)3268 2553 MS
(i)3462 2553 MS
(d)3552 2553 MS
(C)3768 2553 MS
(a)1062 2703 MS
(a)1197 2703 MS
(a)1454 2703 MS
(i)2717 2703 MS
(C)3171 2703 MS ( )3237 2703 MS ( )3263 2703 MS ( )3288 2703 MS ( )3314 2703 MS ( )3339 2703 MS ( )3365 2703 MS ( )3390 2703 MS ( )3416 2703 MS ( )3441 2703 MS
[66.5 0 0 -66.5 0 0]/Helvetica MF
(i)842 2586 MS
(N)3061 2576 MS
(N)3317 2576 MS
(N)3601 2576 MS
(i)1116 2726 MS
(i)1251 2726 MS
(i)1508 2726 MS (N)1522 2726 MS
[91.438 0 0 -91.438 0 0]/Symbol MF
(=)899 2553 MS
(<)983 2553 MS
(\242)1116 2558 MS
(-)1175 2553 MS
(\242)1356 2558 MS
(-)1415 2553 MS
(\242)1996 2558 MS
(-)2068 2553 MS
(\242)2249 2558 MS
(-)2320 2553 MS
(\242)3061 2558 MS
(-)3137 2553 MS
(\242)3318 2558 MS
(-)3394 2553 MS
(>)3896 2553 MS
(=)902 2703 MS
(<)985 2703 MS
(>)1605 2703 MS
(=)2766 2703 MS
[66.5 0 0 -66.5 0 0]/Helvetica MF
(1)1115 2576 MS
(1)1355 2576 MS
(1)1622 2576 MS
(2)2000 2576 MS
(2)2252 2576 MS
(2)2532 2576 MS
(1)1128 2726 MS
(2)1267 2726 MS
[91.438 0 0 -91.438 0 0]/Helvetica MF
(1)2843 2703 MS
(2)2928 2703 MS
(\()1243 2553 MS (\()1273 2553 MS
(*)1519 2553 MS
(\))1664 2553 MS (%)1694 2553 MS
(\))1840 2553 MS
(,)1895 2553 MS
(\()2136 2553 MS (\()2166 2553 MS
(*)2425 2553 MS
(\))2582 2553 MS (%)2612 2553 MS
(\))2758 2553 MS
(,)2813 2553 MS
(.)2859 2553 MS
(.)2886 2553 MS
(.)2914 2553 MS
(,)2960 2553 MS
(\()3205 2553 MS (\()3235 2553 MS
(*)3498 2553 MS
(\))3660 2553 MS (%)3689 2553 MS
(\))3836 2553 MS
(,)1164 2703 MS
(,)1311 2703 MS
(.)1339 2703 MS
(.)1366 2703 MS
(.)1393 2703 MS
(,)1421 2703 MS
(,)2876 2703 MS
(,)2973 2703 MS
(.)3019 2703 MS
(.)3046 2703 MS
(.)3073 2703 MS
(,)3119 2703 MS
( )3467 2703 MS ( )3492 2703 MS ( )3518 2703 MS ( )3543 2703 MS ( )3569 2703 MS ( )3594 2703 MS ( )3620 2703 MS ( )3645 2703 MS ( )3671 2703 MS ( )3697 2703 MS
( )3722 2703 MS ( )3748 2703 MS ( )3773 2703 MS ( )3799 2703 MS ( )3824 2703 MS ( )3850 2703 MS ( )3875 2703 MS ( )3901 2703 MS ( )3926 2703 MS ( )3952 2703 MS
( )3977 2703 MS ( )4003 2703 MS (\()4028 2703 MS (1)4058 2703 MS (0)4109 2703 MS (\))4161 2703 MS
[92 0 0 -92 0 0]/Helvetica MF
(In place of equation 3. Due to the properties of the modulus operator, the receptive field)496 2871 MS
(address components in the above equation are only valid for )496 2977 MS
(s')2981 2977 MS
[58 0 0 -58 0 0]/Helvetica MF
(j)3045 2990 MS
[92 0 0 -92 0 0]/Helvetica MF
(-i*d)3058 2977 MS
[58 0 0 -58 0 0]/Helvetica MF