langtags.html

perl教程

<p>This function, if given a language tag, returns an encoding of it such
that:</p>
<p>* tags representing different languages never get the same encoding.</p>
<p>* tags representing the same language always get the same encoding.</p>
<p>* an encoding of a formally valid language tag always is a string
value that is defined, has length, and is true if considered as a
boolean.</p>
<p>Note that the encoding itself is <strong>not</strong> a formally valid language tag.
Note also that you cannot, currently, go from an encoding back to a
language tag that it's an encoding of.</p>
<p>Note also that you <strong>must</strong> consider the encoded value as atomic; i.e.,
you should not consider it as anything but an opaque, unanalysable
string value.  (The internals of the encoding method may change in
future versions, as the language tagging standard changes over time.)</p>
<p><a href="#item_encode_language_tag"><code>encode_language_tag</code></a> returns undef if given anything other than a
formally valid language tag.</p>
<p>The reason <a href="#item_encode_language_tag"><code>encode_language_tag</code></a> exists is because different language
tags may represent the same language; this is normally treatable with
<a href="#item_same_language_tag"><code>same_language_tag</code></a>, but consider this situation:</p>
<p>You have a data file that expresses greetings in different languages.
Its format is &quot;[language tag]=[how to say 'Hello']&quot;, like:</p>
<pre>
          en-US=Hiho
          fr=Bonjour
          i-mingo=Hau'</pre>
<p>And suppose you write a program that reads that file and then runs as
a daemon, answering client requests that specify a language tag and
then expect the string that says how to greet in that language.  So an
interaction looks like:</p>
<pre>
          greeting-client asks:    fr
          greeting-server answers: Bonjour</pre>
<p>So far so good.  But suppose the way you're implementing this is:</p>
<pre>
          <span class="keyword">my</span> <span class="variable">%greetings</span><span class="operator">;</span>
          <span class="keyword">die</span> <span class="keyword">unless</span> <span class="keyword">open</span><span class="operator">(</span><span class="variable">IN</span><span class="operator">,</span> <span class="string">"&lt;in.dat"</span><span class="operator">);</span>
          <span class="keyword">while</span><span class="operator">(&lt;</span><span class="variable">IN</span><span class="operator">&gt;)</span> <span class="operator">{</span>
            <span class="keyword">chomp</span><span class="operator">;</span>
            <span class="keyword">next</span> <span class="keyword">unless</span> <span class="regex">/^([^=]+)=(.+)/s</span><span class="operator">;</span>
            <span class="keyword">my</span><span class="operator">(</span><span class="variable">$lang</span><span class="operator">,</span> <span class="variable">$expr</span><span class="operator">)</span> <span class="operator">=</span> <span class="operator">(</span><span class="variable">$1</span><span class="operator">,</span> <span class="variable">$2</span><span class="operator">);</span>
            <span class="variable">$greetings</span><span class="operator">{</span><span class="variable">$lang</span><span class="operator">}</span> <span class="operator">=</span> <span class="variable">$expr</span><span class="operator">;</span>
          <span class="operator">}</span>
          <span class="keyword">close</span><span class="operator">(</span><span class="variable">IN</span><span class="operator">);</span>
</pre>
<p>at which point %greetings has the contents:</p>
<pre>
          &quot;en-US&quot;   =&gt; &quot;Hiho&quot;
          &quot;fr&quot;      =&gt; &quot;Bonjour&quot;
          &quot;i-mingo&quot; =&gt; &quot;Hau'&quot;</pre>
<p>And suppose then that you answer client requests for language $wanted
by just looking up $greetings{$wanted}.</p>
<p>If the client asks for &quot;fr&quot;, that will look up successfully in
%greetings, to the value &quot;Bonjour&quot;.  And if the client asks for
&quot;i-mingo&quot;, that will look up successfully in %greetings, to the value
&quot;Hau'&quot;.</p>
<p>But if the client asks for &quot;i-Mingo&quot; or &quot;x-mingo&quot;, or &quot;Fr&quot;, then the
lookup in %greetings fails.  That's the Wrong Thing.</p>
<p>You could instead do lookups on $wanted with:</p>
<pre>
          <span class="keyword">use</span> <span class="variable">I18N::LangTags</span> <span class="string">qw(same_language_tag)</span><span class="operator">;</span>
          <span class="keyword">my</span> <span class="variable">$repsonse</span> <span class="operator">=</span> <span class="string">''</span><span class="operator">;</span>
          <span class="keyword">foreach</span> <span class="keyword">my</span> <span class="variable">$l2</span> <span class="operator">(</span><span class="keyword">keys</span> <span class="variable">%greetings</span><span class="operator">)</span> <span class="operator">{</span>
            <span class="keyword">if</span><span class="operator">(</span><span class="variable">same_language_tag</span><span class="operator">(</span><span class="variable">$wanted</span><span class="operator">,</span> <span class="variable">$l2</span><span class="operator">))</span> <span class="operator">{</span>
              <span class="variable">$response</span> <span class="operator">=</span> <span class="variable">$greetings</span><span class="operator">{</span><span class="variable">$l2</span><span class="operator">}</span><span class="operator">;</span>
              <span class="keyword">last</span><span class="operator">;</span>
            <span class="operator">}</span>
          <span class="operator">}</span>
</pre>
<p>But that's rather inefficient.  A better way to do it is to start your
program with:</p>
<pre>
          <span class="keyword">use</span> <span class="variable">I18N::LangTags</span> <span class="string">qw(encode_language_tag)</span><span class="operator">;</span>
          <span class="keyword">my</span> <span class="variable">%greetings</span><span class="operator">;</span>
          <span class="keyword">die</span> <span class="keyword">unless</span> <span class="keyword">open</span><span class="operator">(</span><span class="variable">IN</span><span class="operator">,</span> <span class="string">"&lt;in.dat"</span><span class="operator">);</span>
          <span class="keyword">while</span><span class="operator">(&lt;</span><span class="variable">IN</span><span class="operator">&gt;)</span> <span class="operator">{</span>
            <span class="keyword">chomp</span><span class="operator">;</span>
            <span class="keyword">next</span> <span class="keyword">unless</span> <span class="regex">/^([^=]+)=(.+)/s</span><span class="operator">;</span>
            <span class="keyword">my</span><span class="operator">(</span><span class="variable">$lang</span><span class="operator">,</span> <span class="variable">$expr</span><span class="operator">)</span> <span class="operator">=</span> <span class="operator">(</span><span class="variable">$1</span><span class="operator">,</span> <span class="variable">$2</span><span class="operator">);</span>
            <span class="variable">$greetings</span><span class="operator">{</span>
                        <span class="variable">encode_language_tag</span><span class="operator">(</span><span class="variable">$lang</span><span class="operator">)</span>
                      <span class="operator">}</span> <span class="operator">=</span> <span class="variable">$expr</span><span class="operator">;</span>
          <span class="operator">}</span>
          <span class="keyword">close</span><span class="operator">(</span><span class="variable">IN</span><span class="operator">);</span>
</pre>
<p>and then just answer client requests for language $wanted by just
looking up</p>
<pre>
          <span class="variable">$greetings</span><span class="operator">{</span><span class="variable">encode_language_tag</span><span class="operator">(</span><span class="variable">$wanted</span><span class="operator">)</span><span class="operator">}</span>
</pre>
<p>And that does the Right Thing.</p>
</li>
<li><strong><a name="item_alternate_language_tags">the function <code>alternate_language_tags($lang1)</code></a></strong>

<p>This function, if given a language tag, returns all language tags that
are alternate forms of this language tag.  (I.e., tags which refer to
the same language.)  This is meant to handle legacy tags caused by
the minor changes in language tag standards over the years; and
the x-/i- alternation is also dealt with.</p>
<p>Note that this function does <em>not</em> try to equate new (and never-used,
and unusable)
ISO639-2 three-letter tags to old (and still in use) ISO639-1
two-letter equivalents -- like &quot;ara&quot; -&gt; &quot;ar&quot; -- because
&quot;ara&quot; has <em>never</em> been in use as an Internet language tag,
and RFC 3066 stipulates that it never should be, since a shorter
tag (&quot;ar&quot;) exists.</p>
<p>Examples:</p>
<pre>
          alternate_language_tags('no-bok')       is ('nb')
          alternate_language_tags('nb')           is ('no-bok')
          alternate_language_tags('he')           is ('iw')
          alternate_language_tags('iw')           is ('he')
          alternate_language_tags('i-hakka')      is ('zh-hakka', 'x-hakka')
          alternate_language_tags('zh-hakka')     is ('i-hakka', 'x-hakka')
          alternate_language_tags('en')           is ()
          alternate_language_tags('x-mingo-tom')  is ('i-mingo-tom')
          alternate_language_tags('x-klikitat')   is ('i-klikitat')
          alternate_language_tags('i-klikitat')   is ('x-klikitat')</pre>
<p>This function returns empty-list if given anything other than a formally
valid language tag.</p>
</li>
<li><strong><a name="item_panic_languages">the function @langs = <code>panic_languages(@accept_languages)</code></a></strong>

<p>This function takes a list of 0 or more language
tags that constitute a given user's Accept-Language list, and
returns a list of tags for <em>other</em> (non-super)
languages that are probably acceptable to the user, to be
used <em>if all else fails</em>.</p>
<p>For example, if a user accepts only 'ca' (Catalan) and
'es' (Spanish), and the documents/interfaces you have
available are just in German, Italian, and Chinese, then
the user will most likely want the Italian one (and not
the Chinese or German one!), instead of getting
nothing.  So <a href="#item_panic_languages"><code>panic_languages('ca', 'es')</code></a> returns
a list containing 'it' (Italian).</p>
<p>English ('en') is <em>always</em> in the return list, but
whether it's at the very end or not depends
on the input languages.  This function works by consulting
an internal table that stipulates what common
languages are &quot;close&quot; to each other.</p>
<p>A useful construct you might consider using is:</p>
<pre>
  <span class="variable">@fallbacks</span> <span class="operator">=</span> <span class="variable">super_languages</span><span class="operator">(</span><span class="variable">@accept_languages</span><span class="operator">);</span>
  <span class="keyword">push</span> <span class="variable">@fallbacks</span><span class="operator">,</span> <span class="variable">panic_languages</span><span class="operator">(</span>
    <span class="variable">@accept_languages</span><span class="operator">,</span> <span class="variable">@fallbacks</span><span class="operator">,</span>
  <span class="operator">);</span>
</pre>
</li>
<li><strong><a name="item_implicate_supers">the function implicate_supers( ...languages... )</a></strong>

<p>This takes a list of strings (which are presumed to be language-tags;
strings that aren't, are ignored); and after each one, this function
inserts super-ordinate forms that don't already appear in the list.
The original list, plus these insertions, is returned.</p>
<p>In other words, it takes this:</p>
<pre>
  pt-br de-DE en-US fr pt-br-janeiro</pre>
<p>and returns this:</p>
<pre>
  pt-br pt de-DE de en-US en fr pt-br-janeiro</pre>
<p>This function is most useful in the idiom</p>
<pre>
  <span class="variable">implicate_supers</span><span class="operator">(</span> <span class="variable">I18N::LangTags::Detect::detect</span><span class="operator">()</span> <span class="operator">);</span>
</pre>
<p>(See <a href="../../lib/I18N/LangTags/Detect.html">the I18N::LangTags::Detect manpage</a>.)</p>
</li>
<li><strong><a name="item_implicate_supers_strictly">the function implicate_supers_strictly( ...languages... )</a></strong>

<p>This works like <a href="#item_implicate_supers"><code>implicate_supers</code></a> except that the implicated
forms are added to the end of the return list.</p>
<p>In other words, implicate_supers_strictly takes a list of strings
(which are presumed to be language-tags; strings that aren't, are
ignored) and after the whole given list, it inserts the super-ordinate forms 
of all given tags, minus any tags that already appear in the input list.</p>
<p>In other words, it takes this:</p>
<pre>
  pt-br de-DE en-US fr pt-br-janeiro</pre>
<p>and returns this:</p>
<pre>
  pt-br de-DE en-US fr pt-br-janeiro pt de en</pre>
<p>The reason this function has &quot;_strictly&quot; in its name is that when
you're processing an Accept-Language list according to the RFCs, if
you interpret the RFCs quite strictly, then you would use
implicate_supers_strictly, but for normal use (i.e., common-sense use,
as far as I'm concerned) you'd use implicate_supers.</p>
</li>
</ul>
<p>
</p>
<hr />
<h1><a name="about_lowercasing">ABOUT LOWERCASING</a></h1>
<p>I've considered making all the above functions that output language
tags return all those tags strictly in lowercase.  Having all your
language tags in lowercase does make some things easier.  But you
might as well just lowercase as you like, or call
<a href="#item_encode_language_tag"><code>encode_language_tag($lang1)</code></a> where appropriate.</p>
<p>
</p>
<hr />
<h1><a name="about_unicode_plaintext_language_tags">ABOUT UNICODE PLAINTEXT LANGUAGE TAGS</a></h1>
<p>In some future version of I18N::LangTags, I plan to include support
for RFC2482-style language tags -- which are basically just normal
language tags with their ASCII characters shifted into Plane 14.</p>
<p>
</p>
<hr />
<h1><a name="see_also">SEE ALSO</a></h1>
<p>* <a href="../../lib/I18N/LangTags/List.html">I18N::LangTags::List</a></p>
<p>* RFC 3066, <code>ftp://ftp.isi.edu/in-notes/rfc3066.txt</code>, &quot;Tags for the
Identification of Languages&quot;.  (Obsoletes RFC 1766)</p>
<p>* RFC 2277, <code>ftp://ftp.isi.edu/in-notes/rfc2277.txt</code>, &quot;IETF Policy on
Character Sets and Languages&quot;.</p>
<p>* RFC 2231, <code>ftp://ftp.isi.edu/in-notes/rfc2231.txt</code>, &quot;MIME Parameter
Value and Encoded Word Extensions: Character Sets, Languages, and
Continuations&quot;.</p>
<p>* RFC 2482, <code>ftp://ftp.isi.edu/in-notes/rfc2482.txt</code>, 
&quot;Language Tagging in Unicode Plain Text&quot;.</p>
<p>* Locale::Codes, in
<code>http://www.perl.com/CPAN/modules/by-module/Locale/</code></p>
<p>* ISO 639-2, &quot;Codes for the representation of names of languages&quot;,
including two-letter and three-letter codes,
<code>http://www.loc.gov/standards/iso639-2/langcodes.html</code></p>
<p>* The IANA list of registered languages (hopefully up-to-date),
<code>http://www.iana.org/assignments/language-tags</code></p>
<p>
</p>
<hr />
<h1><a name="copyright">COPYRIGHT</a></h1>
<p>Copyright (c) 1998+ Sean M. Burke. All rights reserved.</p>
<p>This library is free software; you can redistribute it and/or
modify it under the same terms as Perl itself.</p>
<p>The programs and documentation in this dist are distributed in
the hope that they will be useful, but without any warranty; without
even the implied warranty of merchantability or fitness for a
particular purpose.</p>
<p>
</p>
<hr />
<h1><a name="author">AUTHOR</a></h1>
<p>Sean M. Burke <code>[e-mail]</code></p>

</body>

</html>