readme

very comprehensive example, windows WDM driver development ex The use of drive technology can s

"open" -- clearly wrong.  [Some systems may not even *have* the "udp connected
socket" concept, and netcat in its current form will not work for UDP at all.]
If -z is specified and only one UDP port is probed, netcat's exit status
reflects whether the connection was "open" or "refused" as with TCP.

It may also be that UDP packets are being blocked by filters with no ICMP error
returns, in which case everything will time out and return "open".  This all
sounds backwards, but that's how UDP works.  If you're not sure, try "echo
w00gumz | nc -u -w 2 target 7" to see if you can reach its UDP echo port at
all.  You should have no trouble using a BSD-flavor system to scan for UDP
around your own network, although flooding a target with the high activity that
-z generates will cause it to occasionally drop packets and indicate false
"opens".  A more "correct" way to do this is collect and analyze the ICMP
errors, as does SATAN's "udp_scan" backend, but then again there's no guarantee
that the ICMP gets back to you either.  Udp_scan also does the zero-byte
probes but is excruciatingly careful to calculate its own round-trip timing
average and dynamically set its own response timeouts along with decoding any
ICMP received.  Netcat uses a much sleazier method which is nonetheless quite
effective.  Cisco routers are known to have a "dead time" in between ICMP
responses about unreachable UDP ports, so a fast scan of a cisco will show
almost everything "open".  If you are looking for a specific UDP service, you
can construct a file containing the right bytes to trigger a response from the
other end and send that as standard input.  Netcat will read up to 8K of the
file and send the same data to every UDP port given.  Note that you must use a
timeout in this case [as would any other UDP client application] since the
two-write probe only happens if -z is specified.

Many telnet servers insist on a specific set of option negotiations before
presenting a login banner.  On a raw connection you will see this as small
amount of binary gook.  My attempts to create fixed input bytes to make a
telnetd happy worked some places but failed against newer BSD-flavor ones,
possibly due to timing problems, but there are a couple of much better
workarounds.  First, compile with -DTELNET and use -t if you just want to get
past the option negotiation and talk to something on a telnet port.  You will
still see the binary gook -- in fact you'll see a lot more of it as the options
are responded to behind the scenes.  The telnet responder does NOT update the
total byte count, or show up in the hex dump -- it just responds negatively to
any options read from the incoming data stream.  If you want to use a normal
full-blown telnet to get to something but also want some of netcat's features
involved like settable ports or timeouts, construct a tiny "foo" script:

	#! /bin/sh
	exec nc -otheroptions targethost 23

and then do

	nc -l -p someport -e foo localhost &
	telnet localhost someport

and your telnet should connect transparently through the exec'ed netcat to
the target, using whatever options you supplied in the "foo" script.  Don't
use -t inside the script, or you'll wind up sending *two* option responses.

I've observed inconsistent behavior under some Linuxes [perhaps just older
ones?] when binding in listen mode.  Sometimes netcat binds only to "localhost"
if invoked with no address or port arguments, and sometimes it is unable to
bind to a specific address for listening if something else is already listening
on "any".  The former problem can be worked around by specifying "-s 0.0.0.0",
which will do the right thing despite netcat claiming that it's listening on
[127.0.0.1].  This is a known problem -- for example, there's a mention of it
in the makefile for SOCKS.  On the flip side, binding to localhost and sending
packets to some other machine doesn't work as you'd expect -- they go out with
the source address of the sending interface instead.  The Linux kernel contains
a specific check to ensure that packets from 127.0.0.1 are never sent to the
wire; other kernels may contain similar code.  Linux, of course, *still*
doesn't support source-routing, but they claim that it and many other network
improvements are at least breathing hard.

There are several possible errors associated with making TCP connections, but
to specifically see anything other than "refused", one must wait the full
kernel-defined timeout for a connection to fail.  Netcat's mechanism of
wrapping an alarm timer around the connect prevents the *real* network error
from being returned -- "errno" at that point indicates "interrupted system
call" since the connect attempt was interrupted.  Some old 4.3 BSD kernels
would actually return things like "host unreachable" immediately if that was
the case, but most newer kernels seem to wait the full timeout and *then* pass
back the real error.  Go figure.  In this case, I'd argue that the old way was
better, despite those same kernels generally being the ones that tear down
*established* TCP connections when ICMP-bombed.

Incoming socket options are passed to applications by the kernel in the
kernel's own internal format.  The socket-options structure for source-routing
contains the "first-hop" IP address first, followed by the rest of the real
options list.  The kernel uses this as is when sending reply packets -- the
structure is therefore designed to be more useful to the kernel than to humans,
but the hex dump of it that netcat produces is still useful to have.

Kernels treat source-routing options somewhat oddly, but it sort of makes sense
once one understands what's going on internally.  The options list of addresses
must contain hop1, hop2, ..., destination.  When a source-routed packet is sent
by the kernel [at least BSD], the actual destination address becomes irrelevant
because it is replaced with "hop1", "hop1" is removed from the options list,
and all the other addresses in the list are shifted up to fill the hole.  Thus
the outbound packet is sent from your chosen source address to the first
*gateway*, and the options list now contains hop2, ..., destination.  During
all this address shuffling, the kernel does NOT change the pointer value, which
is why it is useful to be able to set the pointer yourself -- you can construct
some really bizarre return paths, and send your traffic fairly directly to the
target but around some larger loop on the way back.  Some Sun kernels seem to
never flip the source-route around if it contains less than three hops, never
reset the pointer anyway, and tries to send the packet [with options containing
a "completed" source route!!] directly back to the source.  This is way broken,
of course.  [Maybe ipforwarding has to be on?  I haven't had an opportunity to
beat on it thoroughly yet.]

"Credits" section: The original idea for netcat fell out of a long-standing
desire and fruitless search for a tool resembling it and having the same
features.  After reading some other network code and realizing just how many
cool things about sockets could be controlled by the calling user, I started
on the basics and the rest fell together pretty quickly.  Some port-scanning
ideas were taken from Venema/Farmer's SATAN tool kit, and Pluvius' "pscan"
utility.  Healthy amounts of BSD kernel source were perused in an attempt to
dope out socket options and source-route handling; additional help was obtained
from Dave Borman's telnet sources.  The select loop is loosely based on fairly
well-known code from "rsh" and Richard Stevens' "sock" program [which itself is
sort of a "netcat" with more obscure features], with some more paranoid
sanity-checking thrown in to guard against the distinct likelihood that there
are subtleties about such things I still don't understand.  I found the
argument-hiding method cleanly implemented in Barrett's "deslogin"; reading the
line as input allows greater versatility and is much less prone to cause
bizarre problems than the more common trick of overwriting the argv array.
After the first release, several people contributed portability fixes; they are
credited in generic.h and the Makefile.  Lauren Burka inspired the ascii art
for this revised document.  Dean Gaudet at Wired supplied a precursor to
the hex-dump code, and [e-mail] originally experimented with and
supplied code for the telnet-options responder.  Outbound "-e <prog>" resulted
from a need to quietly bypass a firewall installation.  Other suggestions and
patches have rolled in for which I am always grateful, but there are only 26
hours per day and a discussion of feature creep near the end of this document.

Netcat was written with the Russian railroad in mind -- conservatively built
and solid, but it *will* get you there.  While the coding style is fairly
"tight", I have attempted to present it cleanly [keeping *my* lines under 80
characters, dammit] and put in plenty of comments as to why certain things
are done.  Items I know to be questionable are clearly marked with "XXX".
Source code was made to be modified, but determining where to start is
difficult with some of the tangles of spaghetti code that are out there.
Here are some of the major points I feel are worth mentioning about netcat's
internal design, whether or not you agree with my approach.

Except for generic.h, which changes to adapt more platforms, netcat is a single
source file.  This has the distinct advantage of only having to include headers
once and not having to re-declare all my functions in a billion different
places.  I have attempted to contain all the gross who's-got-what-.h-file
things in one small dumping ground.  Functions are placed "dependencies-first",
such that when the compiler runs into the calls later, it already knows the
type and arguments and won't complain.  No function prototyping -- not even the
__P(()) crock -- is used, since it is more portable and a file of this size is
easy enough to check manually.  Each function has a standard-format comment
ahead of it, which is easily found using the regexp " :$".  I freely use gotos.
Loops and if-clauses are made as small and non-nested as possible, and the ends
of same *marked* for clarity [I wish everyone would do this!!].

Large structures and buffers are all malloc()ed up on the fly, slightly larger
than the size asked for and zeroed out.  This reduces the chances of damage
from those "end of the buffer" fencepost errors or runaway pointers escaping
off the end.  These things are permanent per run, so nothing needs to be freed
until the program exits.

File descriptor zero is always expected to be standard input, even if it is
closed.  If a new network descriptor winds up being zero, a different one is
asked for which will be nonzero, and fd zero is simply left kicking around
for the rest of the run.  Why?  Because everything else assumes that stdin is
always zero and "netfd" is always positive.  This may seem silly, but it was a
lot easier to code.  The new fd is obtained directly as a new socket, because
trying to simply dup() a new fd broke subsequent socket-style use of the new fd
under Solaris' stupid streams handling in the socket library.

The catch-all message and error handlers are implemented with an ample list of
phoney arguments to get around various problems with varargs.  Varargs seems
like deliberate obfuscation in the first place, and using it would also
require use of vfprintf() which not all platforms support.  The trailing
sleep in bail() is to allow output to flush, which is sometimes needed if
netcat is already on the other end of a network connection.

The reader may notice that the section that does DNS lookups seems much
gnarlier and more confusing than other parts.  This is NOT MY FAULT.  The
sockaddr and hostent abstractions are an abortion that forces the coder to
deal with it.  Then again, a lot of BSD kernel code looks like similar
struct-pointer hell.  I try to straighten it out somewhat by defining my own
HINF structure, containing names, ascii-format IP addresses, and binary IP
addresses.  I fill this structure exactly once per host argument, and squirrel
everything safely away and handy for whatever wants to reference it later.

Where many other network apps use the FIONBIO ioctl to set non-blocking I/O
on network sockets, netcat uses straightforward blocking I/O everywhere.
This makes everything very lock-step, relying on the network and filesystem
layers to feed in data when needed.  Data read in is completely written out
before any more is fetched.  This may not be quite the right thing to do under
some OSes that don't do timed select() right, but this remains to be seen.

The hexdump routine is written to be as fast as possible, which is why it does
so much work itself instead of just sprintf()ing everything together.  Each
dump line is built into a single buffer and atomically written out using the
lowest level I/O calls.  Further improvements could undoubtedly be made by
using writev() and eliminating all sprintf()s, but it seems to fly right along
as is.  If both exec-a-prog mode and a hexdump file is asked for, the hexdump
flag is deliberately turned off to avoid creating random zero-length files.
Files are opened in "truncate" mode; if you want "append" mode instead, change
the open flags in main().

main() may look a bit hairy, but that's only because it has to go down the
argv list and handle multiple ports, random mode, and exit status.  Efforts
have been made to place a minimum of code inside the getopt() loop.  Any real
work is sent off to functions in what is hopefully a straightforward way.

Obligatory vendor-bash: If "nc" had become a standard utility years ago,
the commercial vendors would have likely packaged it setuid root and with
-DGAPING_SECURITY_HOLE turned on but not documented.  It is hoped that netcat
will aid people in finding and fixing the no-brainer holes of this sort that
keep appearing, by allowing easier experimentation with the "bare metal" of
the network layer.

It could be argued that netcat already has too many features.  I have tried
to avoid "feature creep" by limiting netcat's base functionality only to those
things which are truly relevant to making network connections and the everyday
associated DNS lossage we're used to.  Option switches already have slightly
overloaded functionality.  Random port mode is sort of pushing it.  The
hex-dump feature went in later because it *is* genuinely useful.  The
telnet-responder code *almost* verges on the gratuitous, especially since it
mucks with the data stream, and is left as an optional piece.  Many people have
asked for example "how 'bout adding encryption?" and my response is that such
things should be separate entities that could pipe their data *through* netcat
instead of having their own networking code.  I am therefore not completely
enthusiastic about adding any more features to this thing, although you are
still free to send along any mods you think are useful.

Nonetheless, at this point I think of netcat as my tcp/ip swiss army knife,
and the numerous companion programs and scripts to go with it as duct tape.
Duct tape of course has a light side and a dark side and binds the universe
together, and if I wrap enough of it around what I'm trying to accomplish,
it *will* work.  Alternatively, if netcat is a large hammer, there are many
network protocols that are increasingly looking like nails by now...

_H* 960320 v1.10 RELEASE -- happy spring!