dpid.txt

基于minigui的浏览器. 这是最新版本.

Aug 2003, Jorge Arellano Cid,
           Ferdi Franceschini --
Last update: Dec 2004


                               ------
                                dpid
                               ------

-------------
Nomenclature:
-------------

  dpi:
    generic term referring to dillo's plugin system (version1).

  dpi1: 
    specific term for dillo's plugin spec version 1.
    at: http://www.dillo.org/dpi1.html

  dpi program:
    any plugin program itself.

  dpi framework:
    the code base inside and outside dillo that makes dpi1
    working possible (it doesn't include dpi programs).

  dpip:
    dillo plugin protocol. The HTML/XML like set of command tags
    and information that goes inside the communication sockets.
    Note: not yet fully defined, but functional.
    Note2: it was designed to be extensible.

  dpid:
    dillo plugin daemon.

  server plugin:
    A plugin that is capable of accepting connections on a socket.  Dpid will
    never run more than one instance of a server plugin at a time.

  filter plugin:
    Any program/script that can read or write to stdio.  If you can write a
    shell script you can write one of these (see examples at the end).
    Warning, dpid will run multiple instances of filter plugins if requested.
    This is safe if the plugin only writes to stdout which is what the filter
    type dpis do at the moment.

-----------
About dpid:
-----------

  * dpid is a program which manages dpi connections.
  * dpid is a daemon that serves dillo using unix domain
    sockets (UDS).
  * dpid launches dpi programs and arranges socket communication
    between the dpi program and dillo.

 The  concept  and  motivation  is  similar to that of inetd. The
plugin  manager  (dpid)  listens  for a service request on a Unix
domain  socket  and  returns  the  socket  name  of a plugin that
handles  the service. It also watches sockets of inactive plugins
and starts them when a connection is requested.


-----------------------------------------------------------
What's the problem with managing dpi programs inside dillo?
-----------------------------------------------------------

  That's the other way to handle it, but it started to show some
problems (briefly outlined here):

  * When having two or more running instances of Dillo, one
    should prevail, and take control of dpi managing (but all
    dillos carry the managing code).
  * If the managing dillo exits, it must pass control to another
    instance, or leave it void if there's no other dillo running!
  * The need to synchronise all the running instances of
    dillo arises.
  * If the controlling instance finishes and quits, all the
    dpi-program PIDs are lost.
  * Terminating hanged dpis is hard if it's not done with signals
    (PIDs)
  * Forks can be expensive (Dillo had to fork its dpis).
  * When a managing dillo exits, the new one is no longer the
    parent of the forked dpis.
  * If the Unix domain sockets for the dpis were to be named
    randomly, it gets very hard to recover their names if the
    controlling instance of dillo exits and another must "take
    over" the managing.
  * It increments dillo's core size.
  * ...

  That's why the managing daemon scheme was chosen.


----------------------
What does dpid handle?
----------------------

  It solves all the above mentioned shortcomings and also can do:

  *  Multiple dillos:
     dpid can communicate and serve more than one instance
     of dillo.

  *  Multiple dillo windows:
     two or more windows of the same dillo instance accessing dpis
     at the same time.

  *  Different implementations of the same service
     dpi  programs  ("dpis")  are  just  an  implementation  of a
     service.  There's no problem in having more than one for the
     same service.

  *  Upgrading a service:
     to   a  new  version  or  implementation  without  requiring
     bringing down the dpid or patching dillo's core.


  And  finally,  being  aware  that  this  design can support the
following functions is very helpful:

             SCHEME                      Example
  ------------------------------------------------------------
  * "one demand/one response"         man, preferences, ...
  * "resident while working"          downloads, mp3, ...
  * "resident until TERM signal"      bookmarks, ...
   
  * "one client only"                 cd burner, ...
  * "one client per instance"         man, ...
  * "multiple clients/one instance"   downloads, cookies ...


--------
Features
--------
  * Dpi programs go in: "EPREFIX/dillo/dpi" or "~/.dillo/dpi". The binaries
    are named <name>.dpi as "bookmarks.dpi" and <name>.filter.dpi as in
    "hello.filter.dpi". The ".filter" plugins simply read and write to stdio
    and can be implemented with a shell script easily.
  * Register/update/remove dpis from list of available dpis when a
    <dpi cmd='register_all'> is received.
  * dpid terminates when it receives a <dpi cmd='DpiBye'> command.
  * dpis can be terminated with a <dpi cmd='DpiBye'> command.
  * dpidc control program for dpid, currently allows register and stop.


-----
todo:
-----

 These features are already designed, waiting for implementation:

  * How to register/update/remove/ individual dpis?
  * How to kill dpis?  (signals)

  How:

  A  useful  and  flexible way is to have a "control program" for
dpid (it avoids having to find its PID among others).

  Let's say:

  dpidc [register | upgrade | stop | ...]

  It  can  talk to a dpid UDS that serves for that (the same that
dillo would use). That way we may also have a dpidc dpi! :-)

  Seriously,  what  I  like from this approach is that it is very
flexible  and  can be implemented incrementally ("dpidc register"
is enough to start).

  It  also  avoids the burden of having to periodically check the
dpis directory structure for changes).

  It  also  lets shell scripts an easy way to do the "dirty" work
of   installing  dpis;  as  is  required  with  distros'  package
systems. 

<note>
  How do we tell a crashed dpi? That's the question.
  We're thinking about using the "lease" concept (as in JINI).
</note>


-----------------
How does it work?
-----------------

o    on startup dpid reads dpidrc for the path to the dpi directory
     (usually EPREFIX/lib/dillo/dpi). ~/.dillo/dpi is scanned first.

o    both directories are scanned for the list of available plugins.
     ~/.dillo/dpi overrides system-wide dpis.

o    ~/.dillo/dpi_socket_dir is then checked for the name of the dpi socket
     directory, if dpi_socket_dir does not exist it will be created.

o    next it creates Unix domain sockets for the available plugins and
     then listens for service requests on its own socket (dpid.srs)
     and for connections to the sockets of inactive plugins.

o    dpid returns the name of a plugin's socket when a client (dillo)
     requests a service.

o    if the requested plugin is a 'server' then
     1) dpid stops watching the socket for activity
     2) forks and starts the plugin
     3) resumes watching the socket when the plugin exits

o    if the requested plugin is a 'filter' then
     1) dpid accepts the connection
     2) duplicates the connection on stdio
     3) forks and starts the plugin
     4) continues to watch the socket for new connections




---------------------------
dpi service process diagram
---------------------------

  These drawings should be worth a thousand words! :)


(I)
     .--- s1 s2 s3 ... sn
     |  
  [dpid]                     [dillo]
     |
     '--- srs

  The dpid is running listening on several sockets.


(II)
     .--- s1 s2 s3 ... sn
     |  
  [dpid]                     [dillo]
     |                          |
     '--- srs ------------------'

  dillo needs a service so it connects to the service request
  socket of the dpid (srs) and asks for the socket name of the
  required plugin (using dpip).


(III)
     .--- s1 s2 s3 ... sn
     |          |
  [dpid]        |            [dillo]
     |          |               |
     '--- srs   '---------------'

  then it connects to that socket (s3, still serviced by dpid!)


(IV)
     .--- s1 s2 s3 ... sn
     |          |
 .[dpid]        |            [dillo]
 .   |          |               |
 .   '--- srs   '---------------'
 .               
 .............[dpi program]

  when s3 has activity (incoming data), dpid forks the dpi
  program for it...


(V)
     .--- s1 s2 (s3) ... sn
     |           
  [dpid]                     [dillo]
     |                          |
     '--- srs   .---------------'
                |
              [dpi program]

  ... and lets it "to take over" the socket.

  Once  there's  a  socket  channel  for dpi and dillo, the whole
communication  process  takes  place until the task is done. When
the dpi program exits, dpid resumes listening on the socket (s3).


-----------------------------------------------
How are the unix-domain-sockets for dpis named?
-----------------------------------------------

  Let's say we have two users, "fred" and "joe".

  When  Fred's  dillo  starts  its  dpid,  the  dpid  creates the
following directory (rwx------):

  /tmp/fred-XXXXXX

  using mkdtemp().

  and saves that filename within "~/.dillo/dpi_socket_dir".

  That  way,  another dillo instance of user Fred can easily find
the running dpid's service request socket at:

  /tmp/fred-XXXXXX/dpid.srs

  (because it is saved in "~/.dillo/dpi_socket_dir").

  Now,  we have a dpi directory per user, and its permissions are
locked  so only the user has access, thus the following directory
tree structure should pose no problems:

  /tmp/fred-XXXXXX/bookmarks
                  /downloads
                  /cookies
                  /ftp
                  ...
                  dpid.srs

  If user Joe starts his dillo, the same happens for him:

  /tmp/joe-XXXXXX/bookmarks
                 /downloads
                 /cookies
                 /ftp
                 ...
                 dpid.srs


  What should dpid do at start time:

  Check if both, ~/.dillo/dpi_socket_dir and its directory, exist
  (it can also check the ownership and permissions).

  If (both exist)
     use them!
  else
     delete ~/.dillo/dpi_socket_dir
     create another /tmp/<user>-XXXXXX directory
     save the new directory name into ~/.dillo/dpi_socket_dir
     (we could also add some paranoid level checks)

  To  some degree, this scheme solves the tmpnam issue, different
users  of  dillo  at  the  same  time,  multiple dillo instances,
polluting   /tmp  (cosmetic),  and  reasonably  accounts  for  an
eventual dillo or dpid crash.

  It has worked very well so far!


--------------------------------
So, how do I make my own plugin?
--------------------------------

  First,  at  least, read the "Developing a dillo plugin" section
of dpi1 spec! :-)

  Note that the dpi1 spec may not be absolutely accurate, but the
main ideas remain.

  Once  you've  got the concepts, contrast them with the drawings
in  this  document.  Once  it all makes sense, start playing with
hello.dpi, you can run it by starting dillo with
        dillo dpi:/hello/
or entering
        dpi:/hello/
as the url.  Then try to understand how it works (use the drawings)
and finally look at its code.

  Really, the order is not that important, what really matters is
to do it all.

  Start  modifying  hello.dpi,  and then some more. When you feel
like  trying new things, review the code of the other plugins for
ideas.

  The  hardest  part  is  to try to modify the dpi framework code
inside  dillo; you have been warned! It already supports a lot of
functionality,  but if you need to do some very custom stuff, try
extending the "chat" command.


---------------------------------
Examples: Simple 'filter' plugins
---------------------------------

  For a quick and dirty introduction to dpis try the following shell scripts.

        #!/bin/sh

        read -d'>' dpip_tag # Read dillo's request

        # Don't forget the empty line after the Content-type
        cat <<EOF
        <dpi cmd='start_send_page' url='dpi:/hi/hi.filter.dpi'>
        Content-type: text 

        EOF

        echo Hi
  
 Of course you should use html in a real application (perl makes this easy).

 A more useful example uses the "si" system info viewer:

        #!/bin/sh
        # si - System Information Viewer

        read -d'>' dpip_tag

        # We don't need to send the Content-type because "si --html" does this
        # for us.
        cat <<EOF
        <dpi cmd='start_send_page' url='dpi:/si/si.dpi.filter'>
        EOF

        si --html

 just make sure that you have si installed or you wont get far.

To try out the examples create two directories for the scripts under your home directory as follows:
        mkdir -p ~/.dillo/dpi/hi
        mkdir -p ~/.dillo/dpi/si

then create the scripts and put them in the dpi service directories so that you end up with
        ~/.dillo/dpi/hi/hi.filter.dpi
        ~/.dillo/dpi/si/si.filter.dpi

Don't forget to make them executable.

If dpid is already running register the new plugins with 
        dpidc register

You can now test them by entering
        dpi:/hi/
or
        dpi:/si/
as the url.  Or simply passing the url to dillo on startup

        dillo dpi:/si/


 You can edit the files in place while dpid is running and reload them in
dillo to see the result, however if you change the file name or add a new
script you must run 'dpidc register'.

WARNING
Multiple instances of a filter plugin may be run concurrently, this could be a
problem if your plugin records data in a file, however it is safe if you simply
write to stdout.  Alternatively you could write a 'server' plugin instead as
they are guaranteed not to run concurrently.
        >>>>>>>>>>>>>>>>>>>>>       <<<<<<<<<<<<<<<<<<<<<