boltzmann.ps

手写识别是模式识别中研究得一个热点

V
0 X
N
427.5 321.73 438.75 319.27 2 L
7 X
V
0 X
N
7 X
90 450 2.25 2.45 416.25 292.27 G
0 X
90 450 2.25 2.45 416.25 292.27 A
416.25 302.09 416.25 294.73 2 L
7 X
V
0 X
N
420.75 314.36 439.41 317.54 2 L
7 X
V
0 X
N
443.25 319.27 465.75 314.36 2 L
7 X
V
0 X
N
440.34 296.46 445.5 304.55 2 L
7 X
V
0 X
N
436.5 307 443.25 307 2 L
7 X
V
0 X
N
447.75 307 465.75 307 2 L
7 X
V
0 X
N
447.75 307 465.75 314.36 2 L
7 X
V
0 X
N
418.5 292.27 436.5 294.73 2 L
7 X
V
0 X
N
441 294.73 465.75 307 2 L
7 X
V
0 X
N
420.09 312.63 432 307 2 L
7 X
V
0 X
N
441 316.82 443.91 308.74 2 L
7 X
V
0 X
N
231.75 280 285.75 334 R
7 X
V
0 X
N
384.65 315.19 396.65 315.08 386.42 308.82 386.7 312.33 4 Y
V
366.75 307 386.7 312.33 2 L
N
384.46 310.31 396 307 384.46 303.69 385.67 307 4 Y
V
366.75 307 385.67 307 2 L
N
385.95 304.57 395.99 298 384 298.25 386.12 301.05 4 Y
V
366.75 307 386.13 301.05 2 L
N
211.56 312.79 231.75 307 211.56 301.21 213.67 307 4 Y
V
204.75 307 213.67 307 2 L
3 H
N
312.75 280 366.75 334 R
7 X
V
0.5 H
0 X
N
0 10 Q
(Feature) 321.75 316 T
(Detection) 321.75 306 T
(Pre) 240.75 316 T
(Processing) 240.75 306 T
(Raw) 177.43 309.18 T
(Input) 174.65 300.17 T
0 0 612 792 C
202.5 127 445.5 199 C
281.25 175.83 288 181.58 288 173.92 281.25 168.17 4 Y
3 X
0 K
V
0.5 H
2 Z
0 X
N
274.5 162.42 281.25 168.17 281.25 160.5 274.5 154.75 4 Y
3 X
V
0 X
N
267.75 149 274.5 154.75 274.5 147.08 267.75 141.33 4 Y
3 X
V
0 X
N
274.5 177.75 281.25 183.5 281.25 175.83 274.5 170.08 4 Y
7 X
V
0 X
N
267.75 141.33 274.5 147.08 274.5 139.42 267.75 133.67 4 Y
7 X
V
0 X
N
267.75 164.33 274.5 170.08 274.5 162.42 267.75 156.67 4 Y
7 X
V
0 X
N
267.75 156.67 274.5 162.42 274.5 154.75 267.75 149 4 Y
7 X
V
0 X
N
274.5 154.75 281.25 160.5 281.25 152.83 274.5 147.08 4 Y
7 X
V
0 X
N
274.5 147.08 281.25 152.83 281.25 145.17 274.5 139.42 4 Y
7 X
V
0 X
N
281.25 160.5 288 166.25 288 158.58 281.25 152.83 4 Y
7 X
V
0 X
N
281.25 168.17 288 173.92 288 166.25 281.25 160.5 4 Y
7 X
V
0 X
N
281.25 183.5 288 189.25 288 181.58 281.25 175.83 4 Y
7 X
V
0 X
N
281.25 152.83 288 158.58 288 150.92 281.25 145.17 4 Y
7 X
V
0 X
N
274.5 170.08 281.25 175.83 281.25 168.17 274.5 162.42 4 Y
7 X
V
0 X
N
288 189.25 294.75 195 294.75 187.33 288 181.58 4 Y
7 X
V
0 X
N
288 173.92 294.75 179.67 294.75 172 288 166.25 4 Y
7 X
V
0 X
N
288 166.25 294.75 172 294.75 164.33 288 158.58 4 Y
7 X
V
0 X
N
288 158.58 294.75 164.33 294.75 156.67 288 150.92 4 Y
7 X
V
0 X
N
267.75 172 274.5 177.75 274.5 170.08 267.75 164.33 4 Y
7 X
V
0 X
N
288 181.58 294.75 187.33 294.75 179.67 288 173.92 4 Y
7 X
V
0 X
N
261 150.92 267.75 156.67 267.75 149 261 143.25 4 Y
7 X
V
0 X
N
261 158.58 267.75 164.33 267.75 156.67 261 150.92 4 Y
7 X
V
0 X
N
261 166.25 267.75 172 267.75 164.33 261 158.58 4 Y
7 X
V
0 X
N
261 135.58 267.75 141.33 267.75 133.67 261 127.91 4 Y
7 X
V
0 X
N
261 143.25 267.75 149 267.75 141.33 261 135.58 4 Y
7 X
V
0 X
N
90 450 1.12 0.96 271.12 155.71 G
90 450 1.12 0.96 271.12 155.71 A
90 450 1.12 0.96 277.88 169.13 G
90 450 1.12 0.96 277.88 169.13 A
90 450 1.12 0.96 284.62 159.54 G
90 450 1.12 0.96 284.62 159.54 A
90 450 1.12 0.96 277.88 153.79 G
90 450 1.12 0.96 277.88 153.79 A
90 450 1.12 0.96 284.62 174.88 G
90 450 1.12 0.96 284.62 174.88 A
90 450 1.12 0.96 277.88 161.46 G
90 450 1.12 0.96 277.88 161.46 A
90 450 1.12 0.96 271.12 148.04 G
90 450 1.12 0.96 271.12 148.04 A
90 450 1.12 0.96 271.12 163.38 G
90 450 1.12 0.96 271.12 163.38 A
90 450 1.12 0.96 284.62 167.21 G
90 450 1.12 0.96 284.62 167.21 A
306 133.67 375.75 175.83 R
N
7 X
90 450 2.25 1.92 306 147.08 G
0 X
90 450 2.25 1.92 306 147.08 A
7 X
90 450 2.25 1.92 306 154.75 G
0 X
90 450 2.25 1.92 306 154.75 A
7 X
90 450 2.25 1.92 306 162.42 G
0 X
90 450 2.25 1.92 306 162.42 A
90 450 2.25 1.92 342 154.75 A
90 450 2.25 1.92 348.75 164.33 A
90 450 2.25 1.92 333 166.25 A
90 450 2.25 1.92 333 147.08 A
90 450 2.25 1.92 324 152.83 A
90 450 2.25 1.92 326.25 160.5 A
90 450 2.25 1.92 346.5 145.17 A
90 450 2.25 1.92 353.25 154.75 A
7 X
90 450 2.25 1.92 375.75 160.5 G
0 X
90 450 2.25 1.92 375.75 160.5 A
7 X
90 450 2.25 1.92 375.75 154.75 G
0 X
90 450 2.25 1.92 375.75 154.75 A
308.25 162.42 330.75 166.25 2 L
N
308.25 154.75 321.75 152.83 2 L
N
308.25 162.42 324 160.5 2 L
N
308.25 147.08 322.41 151.48 2 L
N
326.25 158.58 333 149 2 L
N
334.59 148.44 340.41 153.4 2 L
N
335.25 147.08 344.25 145.17 2 L
N
325.59 151.48 331.41 148.44 2 L
N
335.25 166.25 346.5 164.33 2 L
N
90 450 2.25 1.92 324 143.25 A
324 150.92 324 145.17 2 L
N
328.5 160.5 347.16 162.98 2 L
N
351 164.33 373.5 160.5 2 L
N
348.09 146.52 353.25 152.83 2 L
N
344.25 154.75 351 154.75 2 L
N
355.5 154.75 373.5 154.75 2 L
N
355.5 154.75 373.5 160.5 2 L
N
326.25 143.25 344.25 145.17 2 L
N
348.75 145.17 373.5 154.75 2 L
N
327.84 159.15 339.75 154.75 2 L
N
348.75 162.42 351.66 156.11 2 L
N
271.12 148.04 M
 281.25 133.67 281.25 133.67 288 133.67 D
 294.75 133.67 294.75 133.67 295.88 140.38 D
 297 147.08 297 147.08 303.75 147.08 D
N
271.12 163.38 M
 272.25 183.5 272.25 183.5 279 186.38 D
 285.75 189.25 285.75 189.25 291.38 182.54 D
 297 175.83 297 175.83 297 169.12 D
 297 162.42 297 162.42 303.75 162.42 D
N
271.12 155.71 M
 283.5 141.33 283.5 141.33 288 146.12 D
 292.5 150.92 292.5 150.92 294.75 152.83 D
 297 154.75 297 154.75 303.75 154.75 D
N
0 10 Q
(Feature) 382.5 161.17 T
(Output) 382.5 152.18 T
0 0 612 792 C
FMENDPAGE
%%EndPage: "3" 2
%%Page: "2" 2
612 792 0 FMBEGINPAGE
108 54 540 54 2 L
0.25 H
2 Z
0 X
0 K
N
0 8 Q
(Boltzmann Machines) 108 42.62 T
(December 6, 1993) 294.58 42.62 T
(2) 536 42.62 T
1 10 Q
(FIGURE 1. An arti\336cial neur) 263.08 599.08 T
(on) 388.11 599.08 T
1 14 Q
(4.1  T) 108 568.42 T
(opology) 140.53 568.42 T
0 12 Q
-0.31 (The topology for a Boltzmann machine is unlike that of older network con\336gurations such) 108 541.75 P
(as the Perceptron and Selfridge\325) 108 527.75 T
(s Pandemonium which are typically \322layered\323 to some) 261.22 527.75 T
(extent. The Boltzmann topology has much more freedom and may have many layers,) 108 513.75 T
(interconnections between layers, and loops which ultimately feed back on themselves.) 108 499.75 T
(Hence, a Boltzmann net has a basic) 108 485.75 T
2 F
(feedback) 281.2 485.75 T
0 F
( topology) 323.82 485.75 T
0 10 Q
(1) 369.47 490.55 T
0 12 Q
(.) 374.47 485.75 T
1 16 Q
(5.0  Action) 108 445.08 T
0 12 Q
(Like the Hop\336eld network, the Boltzmann network seeks a state with the lowest ener) 108 417.75 T
(gy) 515.5 417.75 T
(.) 526.71 417.75 T
-0.1 (Each unit decides, based on the connections between it and the surrounding units whether) 108 403.75 P
-0.31 (it should be on or of) 108 389.75 P
-0.31 (f. When the sum of the weights from incoming active units is positive,) 203.19 389.75 P
(then the unit is active, otherwise it is inactive. W) 108 375.75 T
(ith the Boltzmann network, the unit will) 341.7 375.75 T
(not be) 108 361.75 T
2 F
(guaranteed) 140.65 361.75 T
0 F
(to turn on depending on the degree of thermal noise. Eventually) 198.27 361.75 T
(, the) 503.61 361.75 T
(entire net will reach a stable) 108 347.75 T
2 F
(equilibrium) 245.89 347.75 T
0 F
( state. The probability of the unit is speci\336ed by) 301.86 347.75 T
(the probability function:) 108 333.75 T
0 10 Q
(\050EQ 1\051) 512.53 301.38 T
0 12 Q
(where) 108 260.75 T
2 F
(T) 140.3 260.75 T
0 F
( is the level of thermal noise, and) 146.97 260.75 T
2 F
(E) 309.19 260.75 T
0 F
( is the sum of the active connection weights.) 316.52 260.75 T
1 14 Q
(5.1  Learning) 108 227.42 T
0 12 Q
(The procedure used to train a Boltzmann machine consists of two phases. The idea is to) 108 200.75 T
(\322teach\323 the network the relationships between the input and the output.) 108 186.75 T
1 11 Q
(Phase I) 108 161.42 T
3 12 Q
(\245) 108 141.75 T
0 F
(Clamp a binary feature vector onto the inputs.) 121.75 141.75 T
3 F
(\245) 108 121.75 T
0 F
(Clamp the desired binary output vector to the output.) 121.75 121.75 T
108 90 540 110 C
108 98 240 98 2 L
0.25 H
2 Z
0 X
0 K
N
0 0 612 792 C
0 10 Q
0 X
0 K
-0.11 (1.  The other type of basic topology is the) 108 83.33 P
2 F
-0.11 (feed-forwar) 275.92 83.33 P
-0.11 (d) 322.74 83.33 P
0 F
-0.11 ( which includes neural network models such as back-) 327.74 83.33 P
(propagation and the neocognitron.) 108 71.33 T
180 619.75 468 708 C
7 X
0 K
90 450 13.5 13.5 319.8 669.94 G
1 H
2 Z
0 X
90 450 13.5 13.5 319.8 669.94 A
294.77 673.25 306.3 669.95 294.77 666.64 295.97 669.95 4 Y
V
279 669.95 295.97 669.95 2 L
0.5 H
N
296.42 665.1 307.95 661.8 296.42 658.49 297.62 661.8 4 Y
2 X
V
282 661.8 297.62 661.8 2 L
N
296.96 681.81 308.5 678.5 296.96 675.19 298.17 678.5 4 Y
4 X
V
281.5 678.5 298.17 678.5 2 L
N
317.1 670.84 320.7 670.84 2 L
0 X
N
0 7 Q
(Inputs from other neurons) 293.71 697.41 T
(Threshold function) 315.29 631.12 T
(W) 240.61 639.67 T
(eighted) 246.65 639.67 T
(Connections) 237.59 634.54 T
278.63 658.92 285.91 660 280.31 655.23 280.14 657.38 4 Y
V
280.15 657.4 280.15 657.38 2 L
V
N
(Output) 370.8 675.81 T
(\050active/inactive\051) 359.04 669.19 T
326.1 678.04 M
 318.9 678.04 318.9 678.04 318.9 670.84 D
 318.9 663.65 318.9 663.65 311.7 663.65 D
1 H
N
342.46 673.25 354 669.95 342.46 666.64 343.67 669.95 4 Y
V
333.3 669.95 343.67 669.95 2 L
0.5 H
N
308.31 683.21 302.12 681.63 306.53 686.25 306.87 684.4 4 Y
V
306.88 684.38 306.88 684.4 2 L
0 Z
N
325.25 695 306.25 684 2 L
N
256.9 646.6 279.95 657 2 L
V
N
325.11 655.93 321.62 663.88 328.79 658.98 326.4 658.13 4 Y
V
326.4 658.12 326.38 658.12 2 L
N
323.62 661.12 342.62 638.12 2 L
N
0 0 612 792 C
261.88 280.75 358.65 317.75 C
2 12 Q
0 X
0 K
(p) 264.63 301.38 T
0 F
(1) 318.2 308.57 T
(1) 290.21 289.19 T
2 F
(e) 308.79 289.19 T
4 9 Q
(D) 324.79 295.82 T
2 F
(E) 330.81 295.82 T
4 F
(\050) 320.9 295.82 T
(\051) 336.7 295.82 T
2 F
(T) 347.19 295.82 T
4 F
(\244) 343.44 295.82 T
(-) 314.12 295.82 T
4 12 Q
(+) 299.21 289.19 T
(=) 276.63 301.38 T
290.21 303.97 351.94 303.97 2 L
0.33 H
0 Z
N
0 0 612 792 C
FMENDPAGE
%%EndPage: "2" 1
%%Page: "1" 1
612 792 0 FMBEGINPAGE
108 54 540 54 2 L
0.25 H
2 Z
0 X
0 K
N
0 8 Q
(Boltzmann Machines) 108 42.62 T
(December 6, 1993) 294.58 42.62 T
(1) 536 42.62 T
0 24 Q
(Boltzmann Machines) 230.73 704 T
1 12 Q
(By Paul Schermerhorn) 274.03 664 T
(CPSC 599.14 Pr) 258.15 648 T
(esentation #4) 340.89 648 T
0 10 Q
(\322A presentation about Boltzmann Machines and their application to Cursive Script recognition.\323) 140.35 621.33 T
1 16 Q
(1.0  Motivation) 117 581.33 T
0 12 Q
(Figuring out what the dif) 117 554 T
(ferences and similarities in handwriting is still an unsolved prob-) 236.72 554 T
(lem. Determining precisely what qualities determine \324a-ness\325 in script \324a\325) 117 540 T
(s are typically) 467.41 540 T
(simple tasks for human experts, yet we have insuf) 117 526 T
(\336cient access to our own knowledge) 356.28 526 T
(base to precisely state the rules for determining \324a-ness\325. What we want is to be able to) 117 512 T
(have the computer \336gure these rules out for) 117 498 T
2 F
(itself) 328.84 498 T
0 F
(.) 352.17 498 T
1 16 Q
(2.0  The Hop\336eld Network) 117 457.33 T
0 12 Q
(In 1982, J.J. Hop\336eld came up with a type of network which minimizes) 117 430 T
2 F
(ener) 463.44 430 T
(gy) 484.31 430 T
0 F
(. This) 494.85 430 T
(model is fully deterministic, and used a simple algorithm to stabilize the network to) 117 416 T
-0.11 (regions of local minima. However) 117 402 P
-0.11 (, we desire) 280.31 402 P
2 F
-0.11 (global) 334.61 402 P
0 F
-0.11 ( minima, not local minima, and this is) 365.27 402 P
(not available with the Hop\336eld model.) 117 388 T
1 16 Q
(3.0  The Boltzmann Machine) 117 347.33 T
0 12 Q
-0.22 (The Boltzmann Machine which was developed at CMU by Ackley) 117 320 P
-0.22 (, Hinton and Sejnowski) 434.3 320 P
(in 1985, is an extension of the ideas proposed in the Hop\336eld net. T) 117 306 T
(riggered by a discov-) 441.37 306 T
(ery of Scott Kirkpatrick in 1983, the Boltzmann machine uses an analogue of thermal) 117 292 T
-0.09 (noise which is gradually reduced. This is similar to the process of slowly cooling metal to) 117 278 P
(strengthen it, hence the term \322simulated annealing.\323 Depending on the amount of noise) 117 264 T
(present in the network, each unit will have a certain probability of being active. This is) 117 250 T
(supposed to get around the local minimum problem present in the Hop\336eld model.) 117 236 T
1 16 Q
(4.0  The Neur) 117 195.33 T
(on) 209.99 195.33 T
0 12 Q
-0.49 (Hop\336eld and Boltzmann networks are comprised of individual neurons or \322units\323 as some,) 117 168 P
(perhaps rightly) 117 154 T
(, prefer to call them. All of the neurons are the same, and have the follow-) 188.51 154 T
(ing qualities.) 117 140 T
3 F
(\245) 117 120 T
0 F
(Connection weights between two units are symmetrical.) 130.74 120 T
3 F
(\245) 117 100 T
0 F
(Each element activates depending on its input conditions.) 130.74 100 T
3 F
(\245) 117 80 T
0 F
(They make asynchronous decisions.) 130.74 80 T
FMENDPAGE
%%EndPage: "1" 0
%%Trailer
%%BoundingBox: 0 0 612 792
%%Pages: 4 -1
%%DocumentFonts: Times-Roman
%%+ Times-Bold
%%+ Times-Italic
%%+ Courier-Bold
%%+ Symbol