readme_parallel

< 数据挖掘--实用机器学习技术及java实现> 一书结合数据挖掘和机器学习的知识,作者陈述了自动挖掘模式的基础理论,并且以java语言实现了具有代表性的各类数据挖掘方法.例如:class

=====================================================================
                                                                       
                              ======                                   
                              README                                   
                              ======                                   
                                                                       
                        Weka-Parallel 3.2.3
                            12 Sep 2002
                                                                       
                 Java Programs for Machine Learning 
                                                                       
                        Copyright (C) 2002 
	          Sebastian Celis, David Musicant

                   email:   [e-mail]
                          [e-mail]

             webpage: http://www.mathcs.carleton.edu/weka/
                                                                       
=====================================================================

Contents:
---------

1. About

2. Getting started

3. Source code

4. Memory Issues

5. Credits

6. Submission of code and bug reports

7. Copyright


----------------------------------------------------------------------

1. About:
---------

This version of Weka was created with the intention of being able to run
the cross-validation portion of any given classifier very quickly.
This speed increase is accomplished by simultaneously performing the
necessary calculations using many different machines.


----------------------------------------------------------------------

2. Getting started:
-------------------

=============
INITIAL SETUP
=============

First, install Weka-Parallel on every machine that is to take
part in the distributed calculations.  Do this the exact same way
you would install Weka.  These installation instructions can be found
in the file README included with both Weka and Weka-Parallel.

A Weka-Parallel session is run on a single client machine and a number of
distributed servers. Each server runs software in the background that
listens for incoming requests placed by the client and then fulfills them.
For each computer that is to be used as a distributed server, launch the
software by entering the following line at a command line prompt on that
computer:

java weka.core.DistributedServer <port number>

Every computer with the DistributedServer program running can act as a
server and will listen on a specified port for all incoming work requests.  
Each request that the server receives is given its own thread, thus
allowing many computers to connect to this one server at the same time.  
This threading will also take advantage of multi-processor machines.  If a
server has two processors, alter the configuration file described below by
telling Weka-Parallel to connect to this computer more than once.  This is
done by entering the address on the configuration file twice.

This server program can be launched manually on each computer or can be
placed in a startup script that runs when each computer boots.  A log
of all connections processed from the server software is automatically
sent to the output screen but can be redirected to a text file if the
user so chooses.

==================
CONFIGURATION FILE
==================

In order for Weka to know which computers to distribute work to, a config
file needs to be present at ~/.weka-parallel on UNIX systems and
C:\WINDOWS\.weka-parallel on Windows systems.  The first line of the config
file should look like:

PORT=XXXX

where XXXX is the port number on which each instance of DistributedServer is
listening.  The remainder of the config file should consist of the addresses
of each computer running an instance of DistributedServer with one address
per line.

The configuration file can be created manually or can be created through
Weka-Parallel's GUI in the cross-validation options under the classifiers pane.

========================
RUNNING WEKA IN PARALLEL
========================

When using a command line interface
-----------------------------------

To do cross-validation within Weka in parallel, simply add the -a tag onto
a standard weka command line. For example:

weka.classifiers.j48.J48 -t weather.arff -a


When using the simple GUI
-------------------------

After selecting a dataset and classifier, first press the button next to
cross-validation to alter the cross-validation settings.  Then check the box
marked "Run in parallel" and then hit "OK".  Finally, run the classifier
and the cross-validation will occur in parallel.


----------------------------------------------------------------------

3. Source code:
---------------

The source code for Weka-Parallel is in weka-src.jar.  To
expand it, use the jar utility that's in every Java distribution.

----------------------------------------------------------------------

4. Memory Issues:
-----------------

Most Java virtual machines allocate a maximum amount of memory to run Java
programs.  Because of this, it is not difficult to exceed available memory
by using a large dataset.  This is typically not a difficult problem as
Java will report an out of memory error to the user. The user can can then 
rerun the program with a larger amount of memory.

We run into a bigger problem in Weka-Parallel when one of the remote
servers runs out of memory.  The Java program running on the server will
crash, with no warning sent back to the client. The user must then
manually restart the Java distributed server program on the server
computer.

Another way that the remote servers might run out of memory is if multiple
copies of Weka-Parallel are connecting to the same remote machine at the
same time.  Due to the threaded nature of Weka-Parallel, the server will
connect to both copies of Weka-Parallel simultaneously and thus load in a
copy of each dataset.  If the datasets are large enough to exceed memory
together, then the DistributedServer will crash.

Thus, if you think that you will ever use large datasets, or if you
believe that multiple people running Weka-Parallel will try and use the
same remote machine at the same time, start the remote machine's program
up with the flags that increase the amount of memory that Java allocates.  
For example, with Sun's JDK,

java -mx250000000 -oss250000000 weka.core.DistributedServer <port>

will set the maximum Java heap and stack size of the remote server to
250,000,000 bytes.

Occasionally, you might see the following message when running Weka-
Parallel:

do_ypcall: clnt_call: RPC: Unable to send; errno = No buffer space available

This occurs when the client runs out of memory while sending data to each
of the servers.  While not guaranteed, the program should continue to run 
and output the correct results.  If you do see the above error message, we
recommend you increase the amount of memory allocated to Weka-Parallel.  

----------------------------------------------------------------------

5. Credits:
-----------

David Musicant  - Project advisor and coordinator

Sebastian Celis - weka.core.DistributedServer,
                  weka.classifiers.EvaluationClient,
                  as well as changes in:
                  weka.classifiers.Evaluation,
                  weka.gui.explorer.ClassifierPanel

A special thanks to the original Weka team who have created an
amazing piece of software.

----------------------------------------------------------------------

6. Submission of code and bug reports:
--------------------------------------

If you have written any code that benefits Weka's new parallelization
features, and you think it should be included in this distribution, send
the code to [e-mail].

If you find any bugs, send a fix to the above address.  If that's too
hard, send a bug report instead.

----------------------------------------------------------------------

7. Copyright:
-------------

Weka and Weka-Parallel are distributed under the GNU public license.
Please read the file COPYING that is included with both Weka and
Weka-Parallel.

----------------------------------------------------------------------