📄 perlunitut.1
字号:
.\" Automatically generated by Pod::Man 2.16 (Pod::Simple 3.05).\".\" Standard preamble:.\" ========================================================================.de Sh \" Subsection heading.br.if t .Sp.ne 5.PP\fB\\$1\fR.PP...de Sp \" Vertical space (when we can't use .PP).if t .sp .5v.if n .sp...de Vb \" Begin verbatim text.ft CW.nf.ne \\$1...de Ve \" End verbatim text.ft R.fi...\" Set up some character translations and predefined strings. \*(-- will.\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left.\" double quote, and \*(R" will give a right double quote. \*(C+ will.\" give a nicer C++. Capital omega is used to do unbreakable dashes and.\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff,.\" nothing in troff, for use with C<>..tr \(*W-.ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p'.ie n \{\. ds -- \(*W-. ds PI pi. if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch. if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch. ds L" "". ds R" "". ds C` "". ds C' ""'br\}.el\{\. ds -- \|\(em\|. ds PI \(*p. ds L" ``. ds R" '''br\}.\".\" Escape single quotes in literal strings from groff's Unicode transform..ie \n(.g .ds Aq \(aq.el .ds Aq '.\".\" If the F register is turned on, we'll generate index entries on stderr for.\" titles (.TH), headers (.SH), subsections (.Sh), items (.Ip), and index.\" entries marked with X<> in POD. Of course, you'll have to process the.\" output yourself in some meaningful fashion..ie \nF \{\. de IX. tm Index:\\$1\t\\n%\t"\\$2"... nr % 0. rr F.\}.el \{\. de IX...\}.\".\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2)..\" Fear. Run. Save yourself. No user-serviceable parts.. \" fudge factors for nroff and troff.if n \{\. ds #H 0. ds #V .8m. ds #F .3m. ds #[ \f1. ds #] \fP.\}.if t \{\. ds #H ((1u-(\\\\n(.fu%2u))*.13m). ds #V .6m. ds #F 0. ds #[ \&. ds #] \&.\}. \" simple accents for nroff and troff.if n \{\. ds ' \&. ds ` \&. ds ^ \&. ds , \&. ds ~ ~. ds /.\}.if t \{\. ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u". ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u'. ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u'. ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u'. ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u'. ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u'.\}. \" troff and (daisy-wheel) nroff accents.ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V'.ds 8 \h'\*(#H'\(*b\h'-\*(#H'.ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#].ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H'.ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u'.ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#].ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#].ds ae a\h'-(\w'a'u*4/10)'e.ds Ae A\h'-(\w'A'u*4/10)'E. \" corrections for vroff.if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u'.if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u'. \" for low resolution devices (crt and lpr).if \n(.H>23 .if \n(.V>19 \\{\. ds : e. ds 8 ss. ds o a. ds d- d\h'-1'\(ga. ds D- D\h'-1'\(hy. ds th \o'bp'. ds Th \o'LP'. ds ae ae. ds Ae AE.\}.rm #[ #] #H #V #F C.\" ========================================================================.\".IX Title "PERLUNITUT 1".TH PERLUNITUT 1 "2007-12-18" "perl v5.10.0" "Perl Programmers Reference Guide".\" For nroff, turn off justification. Always turn off hyphenation; it makes.\" way too many mistakes in technical documents..if n .ad l.nh.SH "NAME"perlunitut \- Perl Unicode Tutorial.SH "DESCRIPTION".IX Header "DESCRIPTION"The days of just flinging strings around are over. It's well established thatmodern programs need to be capable of communicating funny accented letters, andthings like euro symbols. This means that programmers need new habits. It'seasy to program Unicode capable software, but it does require discipline to doit right..PPThere's a lot to know about character sets, and text encodings. It's probablybest to spend a full day learning all this, but the basics can be learned inminutes..PPThese are not the very basics, though. It is assumed that you alreadyknow the difference between bytes and characters, and realise (and accept!)that there are many different character sets and encodings, and that yourprogram has to be explicit about them. Recommended reading is \*(L"The AbsoluteMinimum Every Software Developer Absolutely, Positively Must Know About Unicodeand Character Sets (No Excuses!)\*(R" by Joel Spolsky, at<http://joelonsoftware.com/articles/Unicode.html>..PPThis tutorial speaks in rather absolute terms, and provides only a limited viewof the wealth of character string related features that Perl has to offer. Formost projects, this information will probably suffice..Sh "Definitions".IX Subsection "Definitions"It's important to set a few things straight first. This is the most importantpart of this tutorial. This view may conflict with other information that youmay have found on the web, but that's mostly because many sources are wrong..PPYou may have to re-read this entire section a few times....PP\fIUnicode\fR.IX Subsection "Unicode".PP\&\fBUnicode\fR is a character set with room for lots of characters. The ordinalvalue of a character is called a \fBcode point\fR..PPThere are many, many code points, but computers work with bytes, and a byte canhave only 256 values. Unicode has many more characters, so you need a methodto make these accessible..PPUnicode is encoded using several competing encodings, of which \s-1UTF\-8\s0 is themost used. In a Unicode encoding, multiple subsequent bytes can be used tostore a single code point, or simply: character..PP\fI\s-1UTF\-8\s0\fR.IX Subsection "UTF-8".PP\&\fB\s-1UTF\-8\s0\fR is a Unicode encoding. Many people think that Unicode and \s-1UTF\-8\s0 arethe same thing, but they're not. There are more Unicode encodings, but much ofthe world has standardized on \s-1UTF\-8\s0..PP\&\s-1UTF\-8\s0 treats the first 128 codepoints, 0..127, the same as \s-1ASCII\s0. They takeonly one byte per character. All other characters are encoded as two or more(up to six) bytes using a complex scheme. Fortunately, Perl handles this forus, so we don't have to worry about this..PP\fIText strings (character strings)\fR.IX Subsection "Text strings (character strings)".PP\&\fBText strings\fR, or \fBcharacter strings\fR are made of characters. Bytes areirrelevant here, and so are encodings. Each character is just that: thecharacter..PPText strings are also called \fBUnicode strings\fR, because in Perl, every textstring is a Unicode string..PPOn a text string, you would do things like:.PP.Vb 4\& $text =~ s/foo/bar/;\& if ($string =~ /^\ed+$/) { ... }\& $text = ucfirst $text;\& my $character_count = length $text;.Ve.PPThe value of a character (\f(CW\*(C`ord\*(C'\fR, \f(CW\*(C`chr\*(C'\fR) is the corresponding Unicode codepoint..PP\fIBinary strings (byte strings)\fR.IX Subsection "Binary strings (byte strings)".PP\&\fBBinary strings\fR, or \fBbyte strings\fR are made of bytes. Here, you don't havecharacters, just bytes. All communication with the outside world (anythingoutside of your current Perl process) is done in binary..PPOn a binary string, you would do things like:.PP.Vb 4\& my (@length_content) = unpack "(V/a)*", $binary;\& $binary =~ s/\ex00\ex0F/\exFF\exF0/; # for the brave :)\& print {$fh} $binary;\& my $byte_count = length $binary;.Ve.PP\fIEncoding\fR.IX Subsection "Encoding".PP\&\fBEncoding\fR (as a verb) is the conversion from \fItext\fR to \fIbinary\fR. To encode,you have to supply the target encoding, for example \f(CW\*(C`iso\-8859\-1\*(C'\fR or \f(CW\*(C`UTF\-8\*(C'\fR.Some encodings, like the \f(CW\*(C`iso\-8859\*(C'\fR (\*(L"latin\*(R") range, do not support the fullUnicode standard; characters that can't be represented are lost in theconversion..PP\fIDecoding\fR.IX Subsection "Decoding".PP\&\fBDecoding\fR is the conversion from \fIbinary\fR to \fItext\fR. To decode, you have toknow what encoding was used during the encoding phase. And most of all, it mustbe something decodable. It doesn't make much sense to decode a \s-1PNG\s0 image into atext string..PP\fIInternal format\fR.IX Subsection "Internal format".PPPerl has an \fBinternal format\fR, an encoding that it uses to encode text stringsso it can store them in memory. All text strings are in this internal format.In fact, text strings are never in any other format!.PPYou shouldn't worry about what this format is, because conversion isautomatically done when you decode or encode..Sh "Your new toolkit".IX Subsection "Your new toolkit"Add to your standard heading the following line:.PP.Vb 1\& use Encode qw(encode decode);.Ve.PPOr, if you're lazy, just:.PP.Vb 1\& use Encode;.Ve.Sh "I/O flow (the actual 5 minute tutorial)".IX Subsection "I/O flow (the actual 5 minute tutorial)"The typical input/output flow of a program is:.PP.Vb 3\& 1. Receive and decode\& 2. Process\& 3. Encode and output.Ve.PPIf your input is binary, and is supposed to remain binary, you shouldn't decodeit to a text string, of course. But in all other cases, you should decode it..PPDecoding can't happen reliably if you don't know how the data was encoded. Ifyou get to choose, it's a good idea to standardize on \s-1UTF\-8\s0..PP.Vb 3\& my $foo = decode(\*(AqUTF\-8\*(Aq, get \*(Aqhttp://example.com/\*(Aq);\& my $bar = decode(\*(AqISO\-8859\-1\*(Aq, readline STDIN);\& my $xyzzy = decode(\*(AqWindows\-1251\*(Aq, $cgi\->param(\*(Aqfoo\*(Aq));.Ve.PPProcessing happens as you knew before. The only difference is that you're nowusing characters instead of bytes. That's very useful if you use things like\&\f(CW\*(C`substr\*(C'\fR, or \f(CW\*(C`length\*(C'\fR..PPIt's important to realize that there are no bytes in a text string. Of course,Perl has its internal encoding to store the string in memory, but ignore that.If you have to do anything with the number of bytes, it's probably best to movethat part to step 3, just after you've encoded the string. Then you knowexactly how many bytes it will be in the destination string..PPThe syntax for encoding text strings to binary strings is as simple as decoding:.PP.Vb 1\& $body = encode(\*(AqUTF\-8\*(Aq, $body);.Ve.PPIf you needed to know the length of the string in bytes, now's the perfect timefor that. Because \f(CW$body\fR is now a byte string, \f(CW\*(C`length\*(C'\fR will report thenumber of bytes, instead of the number of characters. The number ofcharacters is no longer known, because characters only exist in text strings..PP.Vb 1\& my $byte_count = length $body;.Ve.PPAnd if the protocol you're using supports a way of letting the recipient knowwhich character encoding you used, please help the receiving end by using thatfeature! For example, E\-mail and \s-1HTTP\s0 support \s-1MIME\s0 headers, so you can use the\&\f(CW\*(C`Content\-Type\*(C'\fR header. They can also have \f(CW\*(C`Content\-Length\*(C'\fR to indicate thenumber of \fIbytes\fR, which is always a good idea to supply if the number isknown..PP.Vb 2\& "Content\-Type: text/plain; charset=UTF\-8",\& "Content\-Length: $byte_count".Ve.SH "SUMMARY".IX Header "SUMMARY"Decode everything you receive, encode everything you send out. (If it's textdata.).SH "Q and A (or FAQ)".IX Header "Q and A (or FAQ)"After reading this document, you ought to read perlunifaq too..SH "ACKNOWLEDGEMENTS".IX Header "ACKNOWLEDGEMENTS"Thanks to Johan Vromans from Squirrel Consultancy. His \s-1UTF\-8\s0 rants during theAmsterdam Perl Mongers meetings got me interested and determined to find outhow to use character encodings in Perl in ways that don't break easily..PPThanks to Gerard Goossen from \s-1TTY\s0. His presentation \*(L"\s-1UTF\-8\s0 in the wild\*(R" (DutchPerl Workshop 2006) inspired me to publish my thoughts and write this tutorial..PPThanks to the people who asked about this kind of stuff in several Perl \s-1IRC\s0channels, and have constantly reminded me that a simpler explanation wasneeded..PPThanks to the people who reviewed this document for me, before it went public.They are: Benjamin Smith, Jan-Pieter Cornet, Johan Vromans, Lukas Mai, NathanGray..SH "AUTHOR".IX Header "AUTHOR"Juerd Waalboer <#####@juerd.nl>.SH "SEE ALSO".IX Header "SEE ALSO"perlunifaq, perlunicode, perluniintro, Encode
⌨️ 快捷键说明
复制代码
Ctrl + C
搜索代码
Ctrl + F
全屏模式
F11
切换主题
Ctrl + Shift + D
显示快捷键
?
增大字号
Ctrl + =
减小字号
Ctrl + -