io-strategies.html

SDK FAQ集

programmers are making?</p>

<p>On the other hand, there are well-known applications that do handle
high volumes of traffic with asynchronous sockets. I assume this is due
to very tolerant code, or very well-written code. It probably also helps
that these applications are dedicated servers: they mainly sit in the
background doing their thing, so they don't have a lot of non-Winsock
messages competing for the attention of the program's message loop
code.</p>

<!-- ------------------------------------------------------------------- -->

<h4>Heuristic 4: For high-performance servers, prefer
overlapped I/O.</h4>

<p>Of all the various I/O strategies, overlapped I/O has the highest
performance. (I/O completion ports are even more efficient, but are
nonstandard vis-a-vis Winsock proper, so I don't cover them in the
FAQ.) With careful use of overlapped I/O (and boatloads of memory in the
server!) you can support <a href="../advanced.html#maxsockets">tens of
thousands of connections</a> with a single server. No other I/O strategy
comes close to the scalability of overlapped I/O.</p>

<!-- ------------------------------------------------------------------- -->

<h4>Heuristic 5: To support a moderate number of
connections, consider asynchronous sockets and event objects.</h4>

<p>If your server only has to support a moderate number of
connections<img src="../bitmaps/waist-dot.gif" alt="--" width=14 height=6 hspace=2>say, between 100 and 1000<img src="../bitmaps/waist-dot.gif" alt="--" width=14 height=6 hspace=2>you may not need
overlapped I/O. Overlapped I/O is not easy to program, so if you don't
need its efficiencies, you can save yourself a lot of trouble by using
a simpler I/O strategy.</p>

<p>Programmed correctly, asynchronous sockets are a reasonable choice for
a dedicated server supporting a moderate number of connections. The main
problem with doing this is that many servers don't have a user interface,
and thus no message loop. A server without a UI using asynchronous
sockets would have to create an invisible window solely to support its
asynchronous sockets. If your program already has a user interface,
though, asynchronous sockets can be the least painful way to add a
network server feature to it.</p>

<p>Another reasonable choice for handling a moderate number of connections
is event objects. These are very efficient in and of themselves. The main
problem you run into with them is that you cannot block on more than 64
event objects at a time. To block on more, you need to create multiple
threads, each of which blocks on a subset of the event objects. Before
choosing this method, consider that handling 1024 sockets requires 64
threads. Any time you have many more active threads than you have
processors in the system, you start causing serious performance
problems. Thus, call 1024 sockets a hard practical limit.</p>

<p>One caution: it's very easy to underestimate the number of simultaneous
connections you will get on a public Internet server. It may make
sense to design for massive scalability even if your estimates don't
currently predict thousands of simultaneous clients. On the other hand,
it's becoming clear that usable-but-weak code today always beats wonderful
code next month.</p>

<!-- ------------------------------------------------------------------- -->

<h4>Heuristic 6: Low-traffic servers can use most any
I/O strategy.</h4>

<p>For low-traffic servers, there isn't much call to be super-efficient.
Some servers just don't have to support very many connections, and
if you're deploying on Win9x you're already going to be limited to <a
href="../advanced.html#maxsockets">100 sockets</a> at a time. Suitable
strategies for 1-100 connections are event objects, asynchronous sockets,
and threads with blocking sockets.</p>

<p>We've covered the first two methods already, so let's consider threads
with blocking sockets. This is by far the simplest way to write a server.
You just have a main loop that accepts connections and spins each new
connection off to its own thread, where it's handled with blocking
sockets. Blocking sockets have several advantages. They are efficient,
because when a thread blocks, the operating system immediately lets
other threads run. Also, synchronous code is more straightforward than
equivalent non-synchronous code.</p>

<p>The problem is that this method doesn't scale well at all. Recall the
discussion of event objects: if the number of active threads outnumbers
the number of processors in the system to a great degree, you run into
efficiency problems. So, this method is only suitable for a fairly
small number of connections, or a moderate number of connections that
are mostly idle.</p>

<!-- ------------------------------------------------------------------- -->

<h4>Heuristic 7: Do not block inside a user interface
thread.</h4>

<p>This heuristic sounds more like a straightforward rule of Windows
programming, but I bring it up because most programs are single-threaded.
In a single-threaded GUI program, any time Winsock blocks, buttons can't
be pressed, menus won't pull down, scroll bars won't move, keypresses
are ignored...your UI freezes.</p>

<!-- ------------------------------------------------------------------- -->

<h4>Heuristic 8: For GUI client programs, prefer
asynchronous sockets.</h4>

<p>There are two reasons for this Heuristic:</p>

<ol>

<li>Asynchronous sockets were designed from the start to work well with
GUI programs. You already have a window loop going, and you already have
window management code in the rest of the program. Adding asynchronous
network I/O is about as easy as adding a dialog to your program.<img src="../bitmaps/dot-clear.gif" alt="" width=1 height=30 align=top> <br clear=all>

<li>All of the alternatives require at least one additional thread to
handle the networking in order to satisfy the previous Heuristic. With
asynchronous sockets, you can handle both the network and the UI with
a single thread. Since window messages are handled one at a time in the
order they arrive, everything is automatically synchronized.<img src="../bitmaps/dot-clear.gif" alt="" width=1 height=30 align=top> <br clear=all>

</ol>

<!-- ------------------------------------------------------------------- -->

<h4>Heuristic 9: Threads are rarely helpful in client
programs.</h4>

<p>When a programmer first learns about threads, he is eager to try them
out in his own programs. He sees that they have several advantages, but he
doesn't yet see the drawbacks. Unfortunately for the soon-to-be-educated
newbie, these drawbacks can have very significant consequences.</p>

<p>One real benefit of threads is that a thread doing I/O on a
blocking socket has a linear control flow, and is therefore easier to
understand. Asynchronous code is more spread out, so it is harder to
write and debug.</p>

<p>Another perceived benefit of threads is a kind of encapsulation:
a programmer can split a program up into a number of threads, each of
which has a single well-defined task. But, this is only valid if each
thread is mostly independent from the rest of the program. If not,
the threads will have to share data through a common data structure,
destroying any potential encapsulation.</p>

<p>In the end, the biggest problem with threads is also related to shared
data structures: synchronization. This issue is covered better elsewhere,
so I won't spend many words on it here. In short, synchronization is hard
to get right: poorly-synchronized threads are subject to serialization
delays, context switching overhead, deadlocks, race conditions and
corrupted data. These are hard problems, and for most programs the
benefits are not large enough to make them worth overcoming.</p>

<p>A saner alternative is to use asynchronous I/O. This buys you the
synchronization benefits described in the previous Heuristic. You
can even partition the application in a similar manner to threads by
creating an invisible window for each socket. If you have two different
types of sockets, each socket can have its notifications sent to a
different type of window. In straight API terms it means a separate
<code>WndProc()</code> for each type of socket. In terms of frameworks
like MFC, you can put the code for each type of socket in a different
subclass of <code>CWnd</code>.</p>

<!-- ------------------------------------------------------------------- -->

<h4>Heuristic 10: Use threads only when their effect on
the rest of the program is easily contained.</h4>

<p>The previous Heuristic cautions that threads are often very hard
to program correctly, but the truth is that they are sometimes very
useful. You can make an educated guess about whether threads will improve
the program by doing a bit of design work: is there a clean interface
between each thread and the rest of the program? If so, synchronization
becomes simple. If not, you're going to end up with a mess that crashes
and destroys data unpredictably.</p>

<p>Examples where threads are viable are:</p>

<ol>

<li><b>An FTP server.</b> One way to write an FTP server is to let the
main thread accept the incoming network connections, and send each one
to a separate thread. Then, each thread can process the incoming FTP
commands, send any required replies, and terminate when the session
closes. Because each thread never has to interact with any other, and
they all act alike, this is an ideal application of threads. (But, keep
in mind the previous server-related Heuristics: one thread per client
severely limits your server's scalability.)<img src="../bitmaps/dot-clear.gif" alt="" width=1 height=30 align=top> <br clear=all>

<li><b>A web browser.</b> When you download a file with a modern web
browser, the file comes down in the background, so that you can continue
browsing. That download stream is most likely handled by a dedicated
thread.<img src="../bitmaps/dot-clear.gif" alt="" width=1 height=30 align=top> <br clear=all>

<li><b>An email program.</b> In an email program, the primary focus is
usually on reading and writing email. However, when an email message
needs to be sent, it is best not to interrupt the user's work. You can
send that message with a separate network thread, since the process
affects the rest of the program only minimally.<img src="../bitmaps/dot-clear.gif" alt="" width=1 height=30 align=top> <br clear=all>

<li><b>A stock ticker.</b> Reduced to basics, a stock ticker simply
displays a small amount of continuous real-time data in a pleasing and
useful format. When the amount of network data involved is low, the
thread synchronization overhead becomes negligible. Plus, this kind of
application only has a single data structure that needs protection; the
really big synchronization problems appear when multiple data structures
need to be protected.<img src="../bitmaps/dot-clear.gif" alt="" width=1 height=30 align=top> <br clear=all>

</ol>

<!-- ------------------------------------------------------------------- -->

<h4>Conclusion</h4>

<p>It is my hope that you find these heuristics helpful. Although you
may not agree with each of them, I think that they will at least make you
think about your own choices. Design is a highly subjective enterprise,
and this list is based mainly on my own thoughts and preferences.</p>

<p>Special thanks go to Philippe Jounin for his comments on the 1998
version of this paper. The 2000 version reflects my greater experience,
as well as commentary from David Schwartz and Alun Jones, both of whom
expanded my ideas of the proper way to build a Winsock server.</p>

<p><font size=-1>Copyright &copy; 1998-2000 by Warren Young. All rights
reserved.</font></p>

		</td>
	</tr>
</table>


<!--  ---- Document Footer ----  -->

<hr noshade size=1 color=#404040>

<table cellpadding=5 cellspacing=0 border=0 width=95% align=center> 
	<tr>
		<td align=left>
		    <a href="../articles/impatient.html">&lt;&lt; Winsock for the Impatient</a>
		</td>

		<td align=right>
		    <a href="../articles/effective-tcp.html">Effective TCP/IP &gt;&gt;</a>
		</td>
	</tr>

	<tr>
		<td align=left>
			<i>Last modified on 29 April 2000 at 15:52 UTC-7</i>
		</td>

		<td align=right>
			<font size=-1>Please send corrections to <a href="mailto:[e-mail]">[e-mail]</a>.</font>
		</td>
	</tr>	
</table>	

<table cellpadding=5 cellspacing=0 border=0 width=95% align=center> 
	<tr>
		<td align=left width=33%>
			<font size=-1>
				<a href="../index.html"><b>&lt;</b> Go to the main FAQ page</a>
			</font>
		</td>

		<td width=33%>
			<font size=-1>
			<center>
				<a href="http://www.cyberport.com/~tangent/programming"><b>&lt;&lt;</b> Go to my Programming pages</a>
			</center>
			</font>
		</td>

		<td align=right width=33%>
			<font size=-1>
				<a href="http://www.cyberport.com/~tangent/"><b>&lt;&lt;&lt;</b> Go to my Home Page</a>
			</font>
		</td>
	</tr>	
</table>	


</body>
</html>