machine learning textbook introduction to machine learning (ethem alpaydin).htm

Machine Learning with WEKA: An Introduction （讲义） 关于数据挖掘和机器学习的.

      <LI>10.9.1 Optimal Separating Hyperplane 218 
      <LI>10.9.2 The Nonseparable Case: Soft Margin Hyperplane 221 
      <LI>10.9.3 Kernel Functions 223 
      <LI>10.9.4 Support Vector Machines for Regression 225 </LI></UL>
    <LI>10.10 Notes 227 
    <LI>10.11 Exercises 227 
    <LI>10.12 References 228 </LI></UL>
  <LI><B>11 Multilayer Perceptrons 229</B> 
  <UL>
    <LI>11.1 Introduction 229 
    <UL>
      <LI>11.1.1 Understanding the Brain 230 
      <LI>11.1.2 Neural Networks as a Paradigm for Parallel Processing 231 
    </LI></UL>
    <LI>11.2 The Perceptron 233 
    <LI>11.3 Training a Perceptron 236 
    <LI>11.4 Learning Boolean Functions 239 
    <LI>11.5 Multilayer Perceptrons 241 
    <LI>11.6 MLP as a Universal Approximator 244 
    <LI>11.7 Backpropagation Algorithm 245 
    <UL>
      <LI>11.7.1 Nonlinear Regression 246 
      <LI>11.7.2 Two-Class Discrimination 248 
      <LI>11.7.3 Multiclass Discrimination 250 
      <LI>11.7.4 Multiple Hidden Layers 252 </LI></UL>
    <LI>11.8 Training Procedures 252 
    <UL>
      <LI>11.8.1 Improving Convergence 252 
      <UL>
        <LI>Momentum 253 
        <LI>Adaptive Learning Rate 253 </LI></UL>
      <LI>11.8.2 Overtraining 253 
      <LI>11.8.3 Structuring the Network 254 
      <LI>11.8.4 Hints 257 </LI></UL>
    <LI>11.9 Tuning the Network Size 259 
    <LI>11.10 Bayesian View of Learning 262 
    <LI>11.11 Dimensionality Reduction 263 
    <LI>11.12 Learning Time 266 
    <UL>
      <LI>11.12.1 Time Delay Neural Networks 266 
      <LI>11.12.2 Recurrent Networks 267 </LI></UL>
    <LI>11.13 Notes 268 
    <LI>11.14 Exercises 270 
    <LI>11.15 References 271 </LI></UL>
  <LI><B>12 Local Models 275</B> 
  <UL>
    <LI>12.1 Introduction 275 
    <LI>12.2 Competitive Learning 276 
    <UL>
      <LI>12.2.1 Online <I>k</I>-Means 276 
      <LI>12.2.2 Adaptive Resonance Theory 281 
      <LI>12.2.3 Self-Organizing Maps 282 </LI></UL>
    <LI>12.3 Radial Basis Functions 284 
    <LI>12.4 Incorporating Rule-Based Knowledge 290 
    <LI>12.5 Normalized Basis Functions 291 
    <LI>12.6 Competitive Basis Functions 293 
    <LI>12.7 Learning Vector Quantization 296 
    <LI>12.8 Mixture of Experts 296 
    <UL>
      <LI>12.8.1 Cooperative Experts 299 
      <LI>12.8.2 Competitive Experts 300 </LI></UL>
    <LI>12.9 Hierarchical Mixture of Experts 300 
    <LI>12.10 Notes 301 
    <LI>12.11 Exercises 302 
    <LI>12.12 References 302 </LI></UL>
  <LI><B>13 Hidden Markov Models 305</B> 
  <UL>
    <LI>13.1 Introduction 305 
    <LI>13.2 Discrete Markov Processes 306 
    <LI>13.3 Hidden Markov Models 309 
    <LI>13.4 Three Basic Problems of HMMs 311 
    <LI>13.5 Evaluation Problem 311 
    <LI>13.6 Finding the State Sequence 315 
    <LI>13.7 Learning Model Parameters 317 
    <LI>13.8 Continuous Observations 320 
    <LI>13.9 The HMM with Input 321 
    <LI>13.10 Model Selection in HMM 322 
    <LI>13.11 Notes 323 
    <LI>13.12 Exercises 325 
    <LI>13.13 References 325 </LI></UL>
  <LI><B>14 Assessing and Comparing Classification Algorithms 327</B> 
  <UL>
    <LI>14.1 Introduction 327 
    <LI>14.2 Cross-Validation and Resampling Methods 330 
    <UL>
      <LI>14.2.1 <I>K</I>-Fold Cross-Validation 331 
      <LI>14.2.2 5x2 Cross-Validation 331 
      <LI>14.2.3 Bootstrapping 332 </LI></UL>
    <LI>14.3 Measuring Error 333 
    <LI>14.4 Interval Estimation 334 
    <LI>14.5 Hypothesis Testing 338 
    <LI>14.6 Assessing a Classification Algorithm's Performance 339 
    <UL>
      <LI>14.6.1 Binomial Test 340 
      <LI>14.6.2 Approximate Normal Test 341 
      <LI>14.6.3 Paired <I>t</I> Test 341 </LI></UL>
    <LI>14.7 Comparing Two Classification Algorithms 341 
    <UL>
      <LI>14.7.1 McNemar's Test 342 
      <LI>14.7.2 <I>K</I>-Fold Cross-Validated Paired <I>t</I> Test 342 
      <LI>14.7.3 5x2 cv Paired <I>t</I> Test 343 
      <LI>14.7.4 5x2 cv Paired <I>F</I> Test 344 </LI></UL>
    <LI>14.8 Comparing Multiple Classification Algorithms: Analysis of Variance 
    345 
    <LI>14.9 Notes 348 
    <LI>14.10 Exercises 349 
    <LI>14.11 References 350 </LI></UL>
  <LI><B>15 Combining Multiple Learners 351</B> 
  <UL>
    <LI>15.1 Rationale 351 
    <LI>15.2 Voting 354 
    <LI>15.3 Error-Correcting Output Codes 357 
    <LI>15.4 Bagging 360 
    <LI>15.5 Boosting 360 
    <LI>15.6 Mixture of Experts Revisited 363 
    <LI>15.7 Stacked Generalization 364 
    <LI>15.8 Cascading 366 
    <LI>15.9 Notes 368 
    <LI>15.10 Exercises 369 
    <LI>15.11 References 370 </LI></UL>
  <LI><B>16 Reinforcement Learning 373</B> 
  <UL>
    <LI>16.1 Introduction 373 
    <LI>16.2 Single State Case: <I>K</I>-Armed Bandit 375 
    <LI>16.3 Elements of Reinforcement Learning 376 
    <LI>16.4 Model-Based Learning 379 
    <UL>
      <LI>16.4.1 Value Iteration 379 
      <LI>16.4.2 Policy Iteration 380 </LI></UL>
    <LI>16.5 Temporal Difference Learning 380 
    <UL>
      <LI>16.5.1 Exploration Strategies 381 
      <LI>16.5.2 Deterministic Rewards and Actions 382 
      <LI>16.5.3 Nondeterministic Rewards and Actions 383 
      <LI>16.5.4 Eligibility Traces 385 </LI></UL>
    <LI>16.6 Generalization 387 
    <LI>16.7 Partially Observable States 389 
    <LI>16.8 Notes 391 
    <LI>16.9 Exercises 393 
    <LI>16.10 References 394 </LI></UL>
  <LI><B>A Probability 397</B> 
  <UL>
    <LI>A.1 Elements of Probability 397 
    <UL>
      <LI>A.1.1 Axioms of Probability 398 
      <LI>A.1.2 Conditional Probability 398 </LI></UL>
    <LI>A.2 Random Variables 399 
    <UL>
      <LI>A.2.1 Probability Distribution and Density Functions 399 
      <LI>A.2.2 Joint Distribution and Density Functions 400 
      <LI>A.2.3 Conditional Distributions 400 
      <LI>A.2.4 Bayes' Rule 401 
      <LI>A.2.5 Expectation 401 
      <LI>A.2.6 Variance 402 
      <LI>A.2.7 Weak Law of Large Numbers 403 </LI></UL>
    <LI>A.3 Special Random Variables 403 
    <UL>
      <LI>A.3.1 Bernoulli Distribution 403 
      <LI>A.3.2 Binomial Distribution 404 
      <LI>A.3.3 Multinomial Distribution 404 
      <LI>A.3.4 Uniform Distribution 404 
      <LI>A.3.5 Normal (Gaussian) Distribution 405 
      <LI>A.3.6 Chi-Square Distribution 406 
      <LI>A.3.7 <I>t</I> Distribution 407 
      <LI>A.3.8 <I>F</I> Distribution 407 </LI></UL>
    <LI>A.4 References 407 </LI></UL>
  <LI><A 
  href="http://mitpress.mit.edu/books/chapters/0262012111index1.pdf">Index 
  409</A> </LI></UL>
<P><A name=revs><B>Reviews:</B></A> 
<UL>
  <LI><A 
  href="http://www.reviews.com/review/review_review.cfm?review_id=130914">ACM 
  Computing Reviews (2005) by L. State</A> <A 
  href="http://www.cmpe.boun.edu.tr/~ethem/i2ml/acm-review.txt">(text copy)</A> 
  <LI><A 
  href="http://www.amazon.com/exec/obidos/tg/detail/-/0262012111?v=glance">Amazon 
  (US) reviews</A> 
  <LI><A href="http://chemeducator.org/bibs/0010002/1020163mr.htm">The Chemical 
  Educator Vol 10:2 (2005) by H Cartwright</A> <A 
  href="http://www.cmpe.boun.edu.tr/~ethem/i2ml/1020163mr.htm">(html copy)</A> 
  <LI><A href="http://www.elsevier.com/locate/jmp">Journal of Mathematical 
  Psychology Vol 49 (2005) 423-424 Telegraphic review by R A Chechile</A> <A 
  href="http://www.cmpe.boun.edu.tr/~ethem/i2ml/j_mathpsyche_rev.pdf">(pdf 
  copy)</A> 
  <LI><A 
  href="http://journals.cambridge.org/action/displayIssue?jid=KER&amp;volumeId=20&amp;issueId=04">The 
  Knowledge Engineering Review Vol 20:4 (2006) 431-433 by S Parsons</A> <A 
  href="http://www.cmpe.boun.edu.tr/~ethem/i2ml/ker_review.pdf">(pdf copy)</A> 
  <LI><A 
  href="http://journals.cambridge.org/action/displayIssue?jid=ROB&amp;volumeId=24&amp;issueId=01#">Robotica 
  Vol 24:1 (2006) 143-144 by G F Page</A> <A 
  href="http://www.cmpe.boun.edu.tr/~ethem/i2ml/robotica06_rev.pdf">(pdf 
  copy)</A> </LI></UL>
<P><A name=courses><B>Courses:</B></A> The book is used in the following 
courses, either as the main textbook, or as a reference book. I will be happy to 
be told of others. 
<UL>
  <LI><B>Textbook:</B> 
  <UL>
    <LI><A href="http://www.cs.toronto.edu/~bonner/courses/2005f/csc411/">A 
    Bonner CSC 411 (Fall 2005) U Toronto at Mississauga (CA)</A> 
    <LI><A 
    href="http://140.122.185.120/Courses/2006S-Machine%20Learning%20&amp;%20Data%20Mining/MLDM_main_2006S.htm">B 
    Chen MLDM (Spring 2006) National Taiwan Normal U (TW)</A> 
    <LI><A href="http://www.ittc.ku.edu/~xwchen/machinelearning.htm">X-w Chen 
    EECS 700 (Fall 2006) U Kansas (US)</A> 
    <LI><A 
    href="http://john.cs.olemiss.edu/~ychen/courses/ENGR691F06/index.html">Y 
    Chen ENGR 691/692 (Fall 2006) U Mississippi (US)</A> 
    <LI><A 
    href="http://www.postech.ac.kr/~seungjin/courses/ml/2006/handouts/handout1.pdf">S 
    Choi EECE 515 (Spring 2006) Pohang U of Sci and Tech (POSTECH) (KR)</A> 
    <LI><A href="http://www.cs.williams.edu/~andrea/cs374/">A Danyluk CS374 
    (2005) Williams College (US)</A> 
    <LI><A 
    href="http://divcom.otago.ac.nz/infosci/courses/homepages/info411/">Da Deng 
    INFO 411 (2006) U Otago (NZ)</A> 
    <LI><A 
    href="http://users.wmin.ac.uk/~dracopd/DOCUM/courses/2ait608/ait608.html">D 
    C Dracopoulos 2AIT608 (Spring 2006) U Westminster (UK)</A> 
    <LI><A href="http://www.cs.rutgers.edu/~elgammal/classes/cs536/cs536.html">A 
    Elgammal 198:536 (Fall 2005) Rutgers U (US)</A> 
    <LI><A href="http://www.cs.ualberta.ca/~greiner/C-466/">R Greiner C466/551 
    (2005) UAlberta (CA)</A> 
    <LI><A href="http://www.cse.ucsc.edu/classes/cmps242/Fall05/">D Helmbold 
    CMPS 242 (Fall 2005) UC Santa Cruz (US)</A> 
    <LI><A href="http://gaia.ecs.csus.edu/~mei/219/cs219.html">M Lu CSc 219 
    (Fall 2006) Cal State Sacramento (US)</A> 
    <LI><A 
    href="http://www.cs.ubc.ca/~murphyk/Teaching/CS340-Fall06/index.html">K 
    Murphy CS 340 (Fall 2006) U British Columbia (CA)</A> 
    <LI><A href="http://www.cs.utk.edu/~parker/Courses/CS594-spring06/">L E