libcurl-tutorial.3

THIS IS HTTP CURL Example

why the server behaves the way it does. Include headers in the normal body
output with CURLOPT_HEADER set TRUE.

Of course there are bugs left. We need to get to know about them to be able
to fix them, so we're quite dependent on your bug reports! When you do report
suspected bugs in libcurl, please include as much details you possibly can: a
protocol dump that CURLOPT_VERBOSE produces, library version, as much as
possible of your code that uses libcurl, operating system name and version,
compiler name and version etc.

If CURLOPT_VERBOSE is not enough, you increase the level of debug data your
application receive by using the CURLOPT_DEBUGFUNCTION.

Getting some in-depth knowledge about the protocols involved is never wrong,
and if you're trying to do funny things, you might very well understand
libcurl and how to use it better if you study the appropriate RFC documents
at least briefly.

.SH "Upload Data to a Remote Site"
libcurl tries to keep a protocol independent approach to most transfers, thus
uploading to a remote FTP site is very similar to uploading data to a HTTP
server with a PUT request.

Of course, first you either create an easy handle or you re-use one existing
one. Then you set the URL to operate on just like before. This is the remote
URL, that we now will upload.

Since we write an application, we most likely want libcurl to get the upload
data by asking us for it. To make it do that, we set the read callback and
the custom pointer libcurl will pass to our read callback. The read callback
should have a prototype similar to:

 size_t function(char *bufptr, size_t size, size_t nitems, void *userp);

Where bufptr is the pointer to a buffer we fill in with data to upload and
size*nitems is the size of the buffer and therefore also the maximum amount
of data we can return to libcurl in this call. The 'userp' pointer is the
custom pointer we set to point to a struct of ours to pass private data
between the application and the callback.

 curl_easy_setopt(easyhandle, CURLOPT_READFUNCTION, read_function);

 curl_easy_setopt(easyhandle, CURLOPT_INFILE, &filedata);

Tell libcurl that we want to upload:

 curl_easy_setopt(easyhandle, CURLOPT_UPLOAD, TRUE);

A few protocols won't behave properly when uploads are done without any prior
knowledge of the expected file size. So, set the upload file size using the
CURLOPT_INFILESIZE_LARGE for all known file sizes like this[1]:

.nf
 /* in this example, file_size must be an off_t variable */
 curl_easy_setopt(easyhandle, CURLOPT_INFILESIZE_LARGE, file_size);
.fi

When you call \fIcurl_easy_perform(3)\fP this time, it'll perform all the
necessary operations and when it has invoked the upload it'll call your
supplied callback to get the data to upload. The program should return as much
data as possible in every invoke, as that is likely to make the upload perform
as fast as possible. The callback should return the number of bytes it wrote
in the buffer. Returning 0 will signal the end of the upload.

.SH "Passwords"
Many protocols use or even require that user name and password are provided
to be able to download or upload the data of your choice. libcurl offers
several ways to specify them.

Most protocols support that you specify the name and password in the URL
itself. libcurl will detect this and use them accordingly. This is written
like this:

 protocol://user:[e-mail]/path/

If you need any odd letters in your user name or password, you should enter
them URL encoded, as %XX where XX is a two-digit hexadecimal number.

libcurl also provides options to set various passwords. The user name and
password as shown embedded in the URL can instead get set with the
CURLOPT_USERPWD option. The argument passed to libcurl should be a char * to
a string in the format "user:password:". In a manner like this:

 curl_easy_setopt(easyhandle, CURLOPT_USERPWD, "myname:thesecret");

Another case where name and password might be needed at times, is for those
users who need to authenticate themselves to a proxy they use. libcurl offers
another option for this, the CURLOPT_PROXYUSERPWD. It is used quite similar
to the CURLOPT_USERPWD option like this:

 curl_easy_setopt(easyhandle, CURLOPT_PROXYUSERPWD, "myname:thesecret");
 
There's a long time unix "standard" way of storing ftp user names and
passwords, namely in the $HOME/.netrc file. The file should be made private
so that only the user may read it (see also the "Security Considerations"
chapter), as it might contain the password in plain text. libcurl has the
ability to use this file to figure out what set of user name and password to
use for a particular host. As an extension to the normal functionality,
libcurl also supports this file for non-FTP protocols such as HTTP. To make
curl use this file, use the CURLOPT_NETRC option:

 curl_easy_setopt(easyhandle, CURLOPT_NETRC, TRUE);

And a very basic example of how such a .netrc file may look like:

.nf
 machine myhost.mydomain.com
 login userlogin
 password secretword
.fi

All these examples have been cases where the password has been optional, or
at least you could leave it out and have libcurl attempt to do its job
without it. There are times when the password isn't optional, like when
you're using an SSL private key for secure transfers.

To pass the known private key password to libcurl:

 curl_easy_setopt(easyhandle, CURLOPT_KEYPASSWD, "keypassword");

.SH "HTTP Authentication"
The previous chapter showed how to set user name and password for getting
URLs that require authentication. When using the HTTP protocol, there are
many different ways a client can provide those credentials to the server and
you can control what way libcurl will (attempt to) use. The default HTTP
authentication method is called 'Basic', which is sending the name and
password in clear-text in the HTTP request, base64-encoded. This is insecure.

At the time of this writing libcurl can be built to use: Basic, Digest, NTLM,
Negotiate, GSS-Negotiate and SPNEGO. You can tell libcurl which one to use
with CURLOPT_HTTPAUTH as in:

 curl_easy_setopt(easyhandle, CURLOPT_HTTPAUTH, CURLAUTH_DIGEST);

And when you send authentication to a proxy, you can also set authentication
type the same way but instead with CURLOPT_PROXYAUTH:

 curl_easy_setopt(easyhandle, CURLOPT_PROXYAUTH, CURLAUTH_NTLM);

Both these options allow you to set multiple types (by ORing them together),
to make libcurl pick the most secure one out of the types the server/proxy
claims to support. This method does however add a round-trip since libcurl
must first ask the server what it supports:

 curl_easy_setopt(easyhandle, CURLOPT_HTTPAUTH,
 CURLAUTH_DIGEST|CURLAUTH_BASIC);

For convenience, you can use the 'CURLAUTH_ANY' define (instead of a list
with specific types) which allows libcurl to use whatever method it wants.

When asking for multiple types, libcurl will pick the available one it
considers "best" in its own internal order of preference.

.SH "HTTP POSTing"
We get many questions regarding how to issue HTTP POSTs with libcurl the
proper way. This chapter will thus include examples using both different
versions of HTTP POST that libcurl supports.

The first version is the simple POST, the most common version, that most HTML
pages using the <form> tag uses. We provide a pointer to the data and tell
libcurl to post it all to the remote site:

.nf
    char *data="name=daniel&project=curl";
    curl_easy_setopt(easyhandle, CURLOPT_POSTFIELDS, data);
    curl_easy_setopt(easyhandle, CURLOPT_URL, "http://posthere.com/");

    curl_easy_perform(easyhandle); /* post away! */
.fi

Simple enough, huh? Since you set the POST options with the
CURLOPT_POSTFIELDS, this automatically switches the handle to use POST in the
upcoming request.

Ok, so what if you want to post binary data that also requires you to set the
Content-Type: header of the post? Well, binary posts prevents libcurl from
being able to do strlen() on the data to figure out the size, so therefore we
must tell libcurl the size of the post data. Setting headers in libcurl
requests are done in a generic way, by building a list of our own headers and
then passing that list to libcurl.

.nf
 struct curl_slist *headers=NULL;
 headers = curl_slist_append(headers, "Content-Type: text/xml");

 /* post binary data */
 curl_easy_setopt(easyhandle, CURLOPT_POSTFIELDS, binaryptr);

 /* set the size of the postfields data */
 curl_easy_setopt(easyhandle, CURLOPT_POSTFIELDSIZE, 23);

 /* pass our list of custom made headers */
 curl_easy_setopt(easyhandle, CURLOPT_HTTPHEADER, headers);

 curl_easy_perform(easyhandle); /* post away! */

 curl_slist_free_all(headers); /* free the header list */
.fi

While the simple examples above cover the majority of all cases where HTTP
POST operations are required, they don't do multi-part formposts. Multi-part
formposts were introduced as a better way to post (possibly large) binary data
and was first documented in the RFC1867. They're called multi-part because
they're built by a chain of parts, each being a single unit. Each part has its
own name and contents. You can in fact create and post a multi-part formpost
with the regular libcurl POST support described above, but that would require
that you build a formpost yourself and provide to libcurl. To make that
easier, libcurl provides \fIcurl_formadd(3)\fP. Using this function, you add
parts to the form. When you're done adding parts, you post the whole form.

The following example sets two simple text parts with plain textual contents,
and then a file with binary contents and upload the whole thing.

.nf
 struct curl_httppost *post=NULL;
 struct curl_httppost *last=NULL;
 curl_formadd(&post, &last,
              CURLFORM_COPYNAME, "name",
              CURLFORM_COPYCONTENTS, "daniel", CURLFORM_END);
 curl_formadd(&post, &last,
              CURLFORM_COPYNAME, "project",
              CURLFORM_COPYCONTENTS, "curl", CURLFORM_END);
 curl_formadd(&post, &last,
              CURLFORM_COPYNAME, "logotype-image",
              CURLFORM_FILECONTENT, "curl.png", CURLFORM_END);

 /* Set the form info */
 curl_easy_setopt(easyhandle, CURLOPT_HTTPPOST, post);

 curl_easy_perform(easyhandle); /* post away! */

 /* free the post data again */
 curl_formfree(post);
.fi

Multipart formposts are chains of parts using MIME-style separators and
headers. It means that each one of these separate parts get a few headers set
that describe the individual content-type, size etc. To enable your
application to handicraft this formpost even more, libcurl allows you to
supply your own set of custom headers to such an individual form part. You can
of course supply headers to as many parts you like, but this little example
will show how you set headers to one specific part when you add that to the
post handle:

.nf
 struct curl_slist *headers=NULL;
 headers = curl_slist_append(headers, "Content-Type: text/xml");

 curl_formadd(&post, &last,
              CURLFORM_COPYNAME, "logotype-image",
              CURLFORM_FILECONTENT, "curl.xml",
              CURLFORM_CONTENTHEADER, headers,
              CURLFORM_END);

 curl_easy_perform(easyhandle); /* post away! */

 curl_formfree(post); /* free post */
 curl_slist_free_all(post); /* free custom header list */
.fi

Since all options on an easyhandle are "sticky", they remain the same until
changed even if you do call \fIcurl_easy_perform(3)\fP, you may need to tell
curl to go back to a plain GET request if you intend to do such a one as your
next request. You force an easyhandle to back to GET by using the
CURLOPT_HTTPGET option:

 curl_easy_setopt(easyhandle, CURLOPT_HTTPGET, TRUE);

Just setting CURLOPT_POSTFIELDS to "" or NULL will *not* stop libcurl from
doing a POST. It will just make it POST without any data to send!

.SH "Showing Progress"

For historical and traditional reasons, libcurl has a built-in progress meter
that can be switched on and then makes it presents a progress meter in your
terminal.

Switch on the progress meter by, oddly enough, set CURLOPT_NOPROGRESS to
FALSE. This option is set to TRUE by default.

For most applications however, the built-in progress meter is useless and
what instead is interesting is the ability to specify a progress
callback. The function pointer you pass to libcurl will then be called on
irregular intervals with information about the current transfer.

Set the progress callback by using CURLOPT_PROGRESSFUNCTION. And pass a
pointer to a function that matches this prototype:

.nf
 int progress_callback(void *clientp,
                       double dltotal,
                       double dlnow,
                       double ultotal,
                       double ulnow);
.fi

If any of the input arguments is unknown, a 0 will be passed. The first
argument, the 'clientp' is the pointer you pass to libcurl with
CURLOPT_PROGRESSDATA. libcurl won't touch it.