perlunicode.html
来自「perl教程」· HTML 代码 · 共 1,163 行 · 第 1/5 页
HTML
1,163 行
<?xml version="1.0" ?>
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN">
<html>
<head>
<!-- saved from url=(0017)http://localhost/ -->
<script language="JavaScript" src="../../displayToc.js"></script>
<script language="JavaScript" src="../../tocParas.js"></script>
<script language="JavaScript" src="../../tocTab.js"></script>
<link rel="stylesheet" type="text/css" href="../../scineplex.css">
<title>perlunicode - Unicode support in Perl</title>
<link rel="stylesheet" href="../../Active.css" type="text/css" />
<meta http-equiv="content-type" content="text/html; charset=utf-8" />
<link rev="made" href="mailto:" />
</head>
<body>
<script>writelinks('__top__',2);</script>
<h1><a>perlunicode - Unicode support in Perl</a></h1>
<p><a name="__index__"></a></p>
<!-- INDEX BEGIN -->
<ul>
<li><a href="#name">NAME</a></li>
<li><a href="#description">DESCRIPTION</a></li>
<ul>
<li><a href="#important_caveats">Important Caveats</a></li>
<li><a href="#byte_and_character_semantics">Byte and Character Semantics</a></li>
<li><a href="#effects_of_character_semantics">Effects of Character Semantics</a></li>
<li><a href="#scripts">Scripts</a></li>
<li><a href="#blocks">Blocks</a></li>
<li><a href="#userdefined_character_properties">User-Defined Character Properties</a></li>
<li><a href="#character_encodings_for_input_and_output">Character Encodings for Input and Output</a></li>
<li><a href="#unicode_regular_expression_support_level">Unicode Regular Expression Support Level</a></li>
<li><a href="#unicode_encodings">Unicode Encodings</a></li>
<li><a href="#security_implications_of_unicode">Security Implications of Unicode</a></li>
<li><a href="#unicode_in_perl_on_ebcdic">Unicode in Perl on EBCDIC</a></li>
<li><a href="#locales">Locales</a></li>
<li><a href="#when_unicode_does_not_happen">When Unicode Does Not Happen</a></li>
<li><a href="#forcing_unicode_in_perl__or_unforcing_unicode_in_perl_">Forcing Unicode in Perl (Or Unforcing Unicode in Perl)</a></li>
<li><a href="#using_unicode_in_xs">Using Unicode in XS</a></li>
</ul>
<li><a href="#bugs">BUGS</a></li>
<ul>
<li><a href="#interaction_with_locales">Interaction with Locales</a></li>
<li><a href="#interaction_with_extensions">Interaction with Extensions</a></li>
<li><a href="#speed">Speed</a></li>
<li><a href="#porting_code_from_perl5_6_x">Porting code from perl-5.6.X</a></li>
</ul>
<li><a href="#see_also">SEE ALSO</a></li>
</ul>
<!-- INDEX END -->
<hr />
<p>
</p>
<h1><a name="name">NAME</a></h1>
<p>perlunicode - Unicode support in Perl</p>
<p>
</p>
<hr />
<h1><a name="description">DESCRIPTION</a></h1>
<p>
</p>
<h2><a name="important_caveats">Important Caveats</a></h2>
<p>Unicode support is an extensive requirement. While Perl does not
implement the Unicode standard or the accompanying technical reports
from cover to cover, Perl does support many Unicode features.</p>
<dl>
<dt><strong><a name="item_input_and_output_layers">Input and Output Layers</a></strong>
<dd>
<p>Perl knows when a filehandle uses Perl's internal Unicode encodings
(UTF-8, or UTF-EBCDIC if in EBCDIC) if the filehandle is opened with
the ":utf8" layer. Other encodings can be converted to Perl's
encoding on input or from Perl's encoding on output by use of the
":encoding(...)" layer. See <a href="../../lib/open.html">the open manpage</a>.</p>
</dd>
<dd>
<p>To indicate that Perl source itself is using a particular encoding,
see <a href="../../lib/encoding.html">the encoding manpage</a>.</p>
</dd>
</li>
<dt><strong><a name="item_regular_expressions">Regular Expressions</a></strong>
<dd>
<p>The regular expression compiler produces polymorphic opcodes. That is,
the pattern adapts to the data and automatically switches to the Unicode
character scheme when presented with Unicode data--or instead uses
a traditional byte scheme when presented with byte data.</p>
</dd>
</li>
<dt><strong><a name="item_use_utf8_still_needed_to_enable_utf_2d8_2futf_2deb"><code>use utf8</code> still needed to enable UTF-8/UTF-EBCDIC in scripts</a></strong>
<dd>
<p>As a compatibility measure, the <code>use utf8</code> pragma must be explicitly
included to enable recognition of UTF-8 in the Perl scripts themselves
(in string or regular expression literals, or in identifier names) on
ASCII-based machines or to recognize UTF-EBCDIC on EBCDIC-based
machines. <strong>These are the only times when an explicit <code>use utf8</code>
is needed.</strong> See <a href="../../lib/utf8.html">the utf8 manpage</a>.</p>
</dd>
<dd>
<p>You can also use the <code>encoding</code> pragma to change the default encoding
of the data in your script; see <a href="../../lib/encoding.html">the encoding manpage</a>.</p>
</dd>
</li>
<dt><strong><a name="item_bom_2dmarked_scripts_and_utf_2d16_scripts_autodete">BOM-marked scripts and UTF-16 scripts autodetected</a></strong>
<dd>
<p>If a Perl script begins marked with the Unicode BOM (UTF-16LE, UTF16-BE,
or UTF-8), or if the script looks like non-BOM-marked UTF-16 of either
endianness, Perl will correctly read in the script as Unicode.
(BOMless UTF-8 cannot be effectively recognized or differentiated from
ISO 8859-1 or other eight-bit encodings.)</p>
</dd>
</li>
<dt><strong><a name="item_use_encoding_needed_to_upgrade_non_2dlatin_2d1_byt"><code>use encoding</code> needed to upgrade non-Latin-1 byte strings</a></strong>
<dd>
<p>By default, there is a fundamental asymmetry in Perl's unicode model:
implicit upgrading from byte strings to Unicode strings assumes that
they were encoded in <em>ISO 8859-1 (Latin-1)</em>, but Unicode strings are
downgraded with UTF-8 encoding. This happens because the first 256
codepoints in Unicode happens to agree with Latin-1.</p>
</dd>
<dd>
<p>If you wish to interpret byte strings as UTF-8 instead, use the
<code>encoding</code> pragma:</p>
</dd>
<dd>
<pre>
<span class="keyword">use</span> <span class="variable">encoding</span> <span class="string">'utf8'</span><span class="operator">;</span>
</pre>
</dd>
<dd>
<p>See <a href="#byte_and_character_semantics">Byte and Character Semantics</a> for more details.</p>
</dd>
</li>
</dl>
<p>
</p>
<h2><a name="byte_and_character_semantics">Byte and Character Semantics</a></h2>
<p>Beginning with version 5.6, Perl uses logically-wide characters to
represent strings internally.</p>
<p>In future, Perl-level operations will be expected to work with
characters rather than bytes.</p>
<p>However, as an interim compatibility measure, Perl aims to
provide a safe migration path from byte semantics to character
semantics for programs. For operations where Perl can unambiguously
decide that the input data are characters, Perl switches to
character semantics. For operations where this determination cannot
be made without additional information from the user, Perl decides in
favor of compatibility and chooses to use byte semantics.</p>
<p>This behavior preserves compatibility with earlier versions of Perl,
which allowed byte semantics in Perl operations only if
none of the program's inputs were marked as being as source of Unicode
character data. Such data may come from filehandles, from calls to
external programs, from information provided by the system (such as %ENV),
or from literals and constants in the source text.</p>
<p>The <code>bytes</code> pragma will always, regardless of platform, force byte
semantics in a particular lexical scope. See <a href="../../lib/bytes.html">the bytes manpage</a>.</p>
<p>The <code>utf8</code> pragma is primarily a compatibility device that enables
recognition of UTF-(8|EBCDIC) in literals encountered by the parser.
Note that this pragma is only required while Perl defaults to byte
semantics; when character semantics become the default, this pragma
may become a no-op. See <a href="../../lib/utf8.html">the utf8 manpage</a>.</p>
<p>Unless explicitly stated, Perl operators use character semantics
for Unicode data and byte semantics for non-Unicode data.
The decision to use character semantics is made transparently. If
input data comes from a Unicode source--for example, if a character
encoding layer is added to a filehandle or a literal Unicode
string constant appears in a program--character semantics apply.
Otherwise, byte semantics are in effect. The <code>bytes</code> pragma should
be used to force byte semantics on Unicode data.</p>
<p>If strings operating under byte semantics and strings with Unicode
character data are concatenated, the new string will be created by
decoding the byte strings as <em>ISO 8859-1 (Latin-1)</em>, even if the
old Unicode string used EBCDIC. This translation is done without
regard to the system's native 8-bit encoding. To change this for
systems with non-Latin-1 and non-EBCDIC native encodings, use the
<code>encoding</code> pragma. See <a href="../../lib/encoding.html">the encoding manpage</a>.</p>
<p>Under character semantics, many operations that formerly operated on
bytes now operate on characters. A character in Perl is
logically just a number ranging from 0 to 2**31 or so. Larger
characters may encode into longer sequences of bytes internally, but
this internal detail is mostly hidden for Perl code.
See <a href="../../lib/Pod/perluniintro.html">the perluniintro manpage</a> for more.</p>
<p>
</p>
<h2><a name="effects_of_character_semantics">Effects of Character Semantics</a></h2>
<p>Character semantics have the following effects:</p>
<ul>
<li>
<p>Strings--including hash keys--and regular expression patterns may
contain characters that have an ordinal value larger than 255.</p>
<p>If you use a Unicode editor to edit your program, Unicode characters
may occur directly within the literal strings in one of the various
Unicode encodings (UTF-8, UTF-EBCDIC, UCS-2, etc.), but will be recognized
as such and converted to Perl's internal representation only if the
appropriate <a href="../../lib/encoding.html">the encoding manpage</a> is specified.</p>
<p>Unicode characters can also be added to a string by using the
<code>\x{...}</code> notation. The Unicode code for the desired character, in
hexadecimal, should be placed in the braces. For instance, a smiley
face is <code>\x{263A}</code>. This encoding scheme only works for characters
with a code of 0x100 or above.</p>
<p>Additionally, if you</p>
<pre>
<span class="keyword">use</span> <span class="variable">charnames</span> <span class="string">':full'</span><span class="operator">;</span>
</pre>
<p>you can use the <code>\N{...}</code> notation and put the official Unicode
character name within the braces, such as <code>\N{WHITE SMILING FACE}</code>.</p>
</li>
<li>
<p>If an appropriate <a href="../../lib/encoding.html">the encoding manpage</a> is specified, identifiers within the
Perl script may contain Unicode alphanumeric characters, including
ideographs. Perl does not currently attempt to canonicalize variable
names.</p>
</li>
<li>
<p>Regular expressions match characters instead of bytes. "." matches
a character instead of a byte. The <code>\C</code> pattern is provided to force
a match a single byte--a <code>char</code> in C, hence <code>\C</code>.</p>
</li>
<li>
<p>Character classes in regular expressions match characters instead of
bytes and match against the character properties specified in the
⌨️ 快捷键说明
复制代码Ctrl + C
搜索代码Ctrl + F
全屏模式F11
增大字号Ctrl + =
减小字号Ctrl + -
显示快捷键?