overview-ini

iscsi源代码 UNH的progect 有initiator端和target端的源码

OVERVIEW.txt
------------

	vi: set autoindent tabstop=4 shiftwidth=4 :
	this document is written in ordinary ASCII text with tabs set
		to a width of 4.


The UNH iscsi reference initiator is implemented as a loadable module
for the Linux kernel.  This document is the beginnings of a document
to explain the internals of this module.

The login phase will be described in a separate document.  This
document explains the general structure and the internal organization
of the Full Feature Phase


Threads
-------

Once the login phase is complete and the connection to a target has been
established, there are the following threads running:

	1 tx_thread for each session
	1 rx_thread for each connection

Multiple connections per session, and multiple sessions per initiator
are both supported.

The tx_thread does all the sending on all connections belonging to a session.
The rx_thread does all the receiving on one connection.

Most of the time the rx_thread is blocked on a call to sock_recvmsg,
waiting for a PDU to arrive from the target.  Also most of the time
the tx_thread is blocked on a semaphore waiting for an "up" signal that
a new PDU has been enqueued for transmission.  There are 3 sources
for a new PDU:

	1.	the SCSI midlevel calls the iscsi_initiator_queuecommand() function
		with a SCSI CDB to be delivered to the target.  This function will
		create a SCSI Command PDU and enqueue it for the tx_thread.

	2.	the SCSI midlevel calls the iscsi_initiator_abort() function in
		order to abort a previously issued command.  This function will
		create a Task Management PDU and enqueue it for the tx_thread.

	3.	the rx_thread can generate a reply to a PDU it has received.
		At present there are two types of PDUs that can be generated in this
		manner -- a DataOut PDU is generated as a reply to an R2T sent by
		the target, and a NopOut PDU is generated as a reply to a NopIn sent
		by the target (that was not a reply to a previously issued NopOut).

Session-specific signaling semaphores
-------------------------------------

Each session has three signaling semaphores defined in the session structure:

	1.	tx_sem
	2.	tx_done_sem
	3.	task_mgt_sem

tx_sem is initialized to locked when a session is created, and the tx_thread
normally blocks on a call to down_interruptible() on this semaphore.  Whenever
the rx_thread or the functions called by the SCSI midlevel enqueue a new PDU
to be transmitted to the target associated with this session, they do an
up() on the session's tx_sem to wake up the tx_thread (which will cause
the tx_thread to actually transmit the PDU on the proper connection in
the proper order).

tx_done_sem is also initialized to locked when a session is created.  A new
session is created via the /proc interface, so the thread that creates a
session is the one that is processing the write to the /proc interface, which
is handled by our iscsi_initiator_proc_info() function.  After all the
data structures for the new session have been correctly created and
initialized, and the login phase has successfully concluded, the tx_thread
is started.  The /proc thread does a down() on tx_done_sem to wait until
the tx_thread has actually started and initialized itself, at which time
it does the corresponding up() on tx_done_sem.

Once going, the tx_thread does not do another up() on the tx_done_sem until
it terminates itself for any reason.  The function close_session() is called
to terminate a session, which is triggered by two different sources (each
running in its own thread created by the user-level command): if the module
is unloaded using /sbin/rmmod, at which time our iscsi_initiator_release()
function is called; and if the session is terminated by a write to the
/proc interface, as handled by our iscsi_initiator_proc_info() function.
The close_session() function will send a kill signal to the tx_thread and then
block on a down_interruptible() on the tx_done_sem to wait until the tx_thread
shuts itself down cleanly.

task_mgt_sem is also initialized to locked when a session is created.
When the iscsi_initiator_abort() function is called by the SCSI midlevel,
a task management command is queued for the tx_thread, a local timer is
activated to wait for at most 3 seconds for a response from the target,
and then iscsi_initiator_abort() does a down() on the task_mgt_sem for the
session.  If the timer expires before a response is received, the interrupt
service routine for this timer, abort_timer_function(), will do an up()
on the task_mgt_sem.  If a response is received, it will be processed by
rx_task_mgt_rsp(), which also does an up() on the task_mgt_sem for the
session.


Connection-specific signaling semaphores
----------------------------------------

Each connection has one signaling sempahore defined in the connection
structure:

	1.	rx_done_sem

rx_done_sem is initialized to locked when a connection is created, and the
rx_thread does an up() on it when it terminates for any reason.  The function
close_connection() will send a kill signal to the rx_thread and then block on
a down_interruptible() on the rx_done_sem to wait until the rx_thread shuts
itself down cleanly.

rx_done_sem is initialized to locked when a connection is created.  A new
connection is created via the /proc interface, so the thread that creates a
connection is the one that is processing the write to the /proc interface,
which is handled by our iscsi_initiator_proc_info() function.  After all the
data structures for the new connection have been correctly created and
initialized, and the login phase has successfully concluded, the rx_thread
is started.  The /proc thread does a down() on rx_done_sem to wait until
the rx_thread has actually started and initialized itself, at which time
it does the corresponding up() on rx_done_sem.

Once going, the rx_thread does not do another up() on the rx_done_sem until
it terminates itself for any reason.  The function close_connection() is
called to terminate a connection, which is triggered by two different sources
(each running in its own thread created by the user-level command): if the
module is unloaded using /sbin/rmmod, at which time our
iscsi_initiator_release() function is called; and if the connection is
terminated by a write to the /proc interface, as handled by our
iscsi_initiator_proc_info() function.  The close_connection() function will
send a kill signal to the rx_thread and then block on a down_interruptible()
on the rx_done_sem to wait until the rx_thread shuts itself down cleanly.


Timers
------

There are is one local timer created and destroyed for each task management
command, as explained above under the description of the task_mgt_sem.

There is also one session-specific timer called tx_timer.  This is created
and initialized in init_session(), which is called by create_session()
whenever a new session is created.  When it starts up, the tx_thread
calls restart_tx_timer() with the "nop_period" converted to jiffies.
The nop_period is a session-specific variable that can be "forced" by
the iscsi_manage program to a non-zero value which is the nop period in
seconds.  What this means is if there is no activity received on a
connection during one nop period, the initiator will send a NopOut PDU
to the target to "ping" it to see if it is still alive.  This NopOut PDU
has data attached, and a NopIn response from the target with the same
data attached is expected.


restart_tx_timer() activates the session-specific tx_timer with the
period supplied as a parameter, assuming certain other sanity conditions
are met.  When this timer expires, the kernel calls the deal_with_tx_timer()
function, which does an up() on the session-specific tx_sem (thereby
waking the tx_thread if it is blocked) and then calls restart_tx_timer()
again to keep the timer running.  If the tx_thread itself receives an
EAGAIN error from a call to sock_sendmsg(), meaning that data could not
be written to a socket because it was backed up, then the tx_thread
also calls restart_tx_timer() with a value of TX_RESTART_PERIOD (10)
and then blocks again on the tx_sem for that session.  This gives the
TCP/IP sublayers 10 jiffies (100 milliseconds) to clear the buffers
before the tx_thread tries again to send data.


recv_a_message()
----------------

After login has been completed, there is only a single place where
sock_recvmsg() is called by the isci_initiator, and this call is in the
main input routine called recv_iovector().  There are 3 parameters to
recv_iovector():

	1.	A pointer to the connection structure on which to read.
	2.	The number of bytes to read, which MUST be positive.
	3.	The number of io vectors which have been set up in the connection's
		rx_iov array.

recv_iovector() uses the rx_msg area of the connection structure
to set up the message header required by sock_recvmsg(), but the i/o vector
must have been previously set up in the connection's rx_iov array by the
caller to sock_recvmsg().

The caller to recv_iovector() MUST hold the session->sess_lock spinlock prior
to the call, with the flags saved in the lock_flags field for the current
session.  Just before calling sock_recvmsg(), recv_iovector() releases
this lock, thereby releasing the global exclusive access in the
(highly likely) case that the sock_recvmsg() call will block until data
arrives from the target.  When sock_recvmsg() returns, successfully or not,
recv_iovector() gets the session->sess_lock spinlock again to regain global
exclusive access to all the session structures.  If sock_recvmsg() fails or is
interrupted by a signal, recv_iovector() releases the lock and returns a
negative error code.  If sock_recvmsg() succeeds, recv_iovector() returns the
number of bytes received (which will always be equal to the number of bytes
expected, because recv_iovector() will loop until this number has been received
or an error occurs).  It is for this reason that the parameter specifying
the number of bytes to read MUST be positive when recv_iovector() is called.

To summarize, there are 2 possible return values from recv_iovector():

	1.	> 0	Success, io_request_lock is locked
	2.	< 0 Failure, io_request_lock is NOT locked

There are only 2 calls to recv_iovector() in the iscsi initiator:

	1.	From recv_pdu_header() to read the next PDU header.
	2.	From recv_data_in_data() to read data into a SCSI scatter-gather list.


recv_pdu_header()
---------------

This routine is called in order to set up the connection's rx_iov
to read a 48-byte or 52-byte header (if header digests are in use)
into the connections rx_buf area.

recv_data_in_data()
-----------------

This routine is called in order to set up the connection's rx_iov
to read data into scsi buffers.  This routine maps the scsi midlevel's
scatter-gather list into the necessary iovector slots, and takes care
of adding padding and/or the data digest and/or overflow data.



Key Parameters
--------------

1.	The number of key parameters recognized in this implementation of Draft 20
	is given in the file "common/text_param.h" as:

		#define	MAX_CONFIG_PARAMS	28