draft-ietf-idn-utf6-00.txt

bind-3.2.

Internet Engineering Task Force (IETF)                         Mark Welter
INTERNET-DRAFT                                          Brian W. Spolarich
draft-ietf-idn-utf6-00                                         WALID, Inc.
November 16, 2000                                     Expires May 16, 2001


        UTF-6 - Yet Another ASCII-Compatible Encoding for IDN

Status of this memo

This document is an Internet-Draft and is in full conformance with all
provisions of Section 10 of RFC2026.

Internet-Drafts are working documents of the Internet Engineering Task
Force (IETF), its areas, and its working groups. Note that other
groups may also distribute working documents as Internet-Drafts.

Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as "work in progress."

     The list of current Internet-Drafts can be accessed at
     http://www.ietf.org/ietf/1id-abstracts.txt

     The list of Internet-Draft Shadow Directories can be accessed at
     http://www.ietf.org/shadow.html.

The distribution of this document is unlimited.

Copyright (c) The Internet Society (2000).  All Rights Reserved.

Abstract

This document describes a tranformation method for representing
Unicode character codepoints in host name parts in a fashion that is 
completely compatible with the current Domain Name System.  It is 
proposed as a potential candidate for an ASCII-Compatible Encoding (ACE)
for supporting the deployment of an internationalized Domain Name System.
The tranformation method, an extension of the UTF-5 encoding proposed by
Duerst, provides both for more efficient representation of typical Unicode 
sequences while preserving simplicity and readability.  This transformation 
method is deployed as part of the current WALID multilingual domain name 
system implementation, although that status should not necessarily influence 
the evaluation of its merits as a candidate encoding method.


Table of Contents

1.        Introduction
1.1         Terminology
2.        Hostname Part Transformation
2.1         Post-Converted Name Prefix
2.2         Hostname Prepartion
2.3         Definitions
2.4         UTF-6 Encoding
2.4.1         Variable Length Hex Encoding
2.4.2         UTF-6 Compression Algorithm
2.4.3         Forward Transformation Algorithm
2.5         UTF-6 Decoding
2.5.1         Variable Length Hex Decoding
2.5.2         UTF-6 Decompression Algorithm
2.5.3         Reverse Transformation Algorithm
3.        Examples
3.1         'www.walid.com' (in Arabic)
4.        Security Considerations
5.        References


1.  Introduction

UTF-6 describes an encoding scheme of the ISO/IEC 10646 [ISO10646]
character set (whose character code assignments are synchronized
with Unicode [UNICODE3]), and the procedures for using this scheme
to transform host name parts containing Unicode character sequences
into sequences that are compatible with the current DNS protocol
[STD13].  As such, it satisfies the definition of a 'charset' as
defined in [IDNREQ].

1.1  Terminology

The key words "MUST", "SHALL", "REQUIRED", "SHOULD", "RECOMMENDED", and
"MAY" in this document are to be interpreted as described in RFC 2119
[RFC2119].

Hexadecimal values are shown preceded with an "0x". For example,
"0xa1b5" indicates two octets, 0xa1 followed by 0xb5. Binary values are
shown preceded with an "0b". For example, a nine-bit value might be
shown as "0b101101111".

Examples in this document use the notation from the Unicode Standard
[UNICODE3] as well as the ISO 10646 names. For example, the letter "a"
may be represented as either "U+0061" or "LATIN SMALL LETTER A".

UTF-6 converts strings with internationalized characters into
strings of US-ASCII that are acceptable as host name parts in current
DNS host naming usage. The former are called "pre-converted" and the
latter are called "post-converted".  This specification defines both
a forward and reverse transformation algorithm.


2.  Hostname Part Transformation

According to [STD13], hostname parts must be case-insensitive, start and
end with a letter or digit, and contain only letters, digits, and the
hyphen character ("-"). This, of course, excludes most characters used
by non-English speakers, characters, as well as many other characters in 
the ASCII character repertoire. Further, domain name parts must be 
63 octets or shorter in length.


2.1  Post-Converted Name Prefix

This document defines the string 'wq--' as a prefix to identify 
UTF-6-encoded sequences.  For the purposes of comparison in the IDN 
Working Group activities, the 'wq--' prefix should be used solely to 
identify UTF-6 sequences.  However, should this document proceed beyond 
draft status the prefix should be changed to whatever prefix, if any,
is the final consensus of the IDN working group.

Note that the prepending of a fixed identifier sequence is only one
mechanism for differentiating ASCII character encoded international
domain names from 'ordinary' domain names.  One method, as proposed in
[IDNRACE], is to include a character prefix or suffix that does not
appear in any name in any zone file.  A second method is to insert a
domain component which pushes off any international names one or more
levels deeper into the DNS heirarchy.  There are trade-offs between
these two methods which are independent of the Unicode to ASCII
transcoding method finally chosen.  We do not address the international
vs. 'ordinary' name differention issue in this paper.

2.2  Hostname Prepartion

The hostname part is assumed to have at least one character disallowed
by [STD13], and that is has been processed for logically equivalent 
character mapping, filtering of disallowed characters (if any), and 
compatibility composition/decomposition before presentation to the UTF-6 
conversion algorithm.  

While it is possible to invent a transcoding mechanism that relies
on certain Unicode characters being deemed illegal within domain names
and hence available to the transcoding mechanism for improving encoding
efficiency, we feel that such a proposal would complicate matters
excessively.  We also believe that Unicode name preprocessing for
both name resolution and name registration should be considered as s
separate, independent issues, which we will attempt to address in a
separate document.

2.3  Definitions

For clarity:

  'integer' is an unsigned binary quantity;
  'byte' is an 8-bit integer quantity;
  'nibble' is a 4-bit integer quantity.

2.4  UTF-6 Encoding

The idea behind this scheme was to improve on the UTF-5 transformation
algorithm described in [IDNDUERST] by providing a straightforward
compression mechanism.  UTF-6 defines a compression mechanism by
indentifying identical leading byte or nibble values in the pre-converted
string, and using the length of this leading value to select a mask which
can be applied to the pre-converted string.  The resulting post-converted
string is preserves the simplicity and readability of UTF-5 while 
enabling longer sequences to be encoded into a single host name part.

2.4.1  Variable Length Hex Encoding

The variable length hex encoding algorithm was introduced by Duerst in 
[IDNDUERST].  It encodes an integer value in a slight modification of 
traditional hexadecimal notation, the difference being that the most 
significant digit is represented with an alternate set of "digits" 
- -- 'g through 'v' are used to represent 0 through 15.  The result is a 
variable length encoding which can efficiently represent integers of 
arbitrary length. 

The variable length nibble encoding of an integer, C, is defined
as follows:

  1.  Skip over leading non-significant zero nibbles to find I,
      the first significant nibble of c;

  2.  Emit the Ith character of the set [ghijklmopqrstuv];

  3.  Continue from most to least significant, encoding each remaining
      nibble J by emitting the Jth character of the set [0123456789abcdef].

Examples:

  0x1f4c    is encoded as "hf4c"
  0x0624    is encoded as "m24"
  0x0000    is encoded as "g"
  'n'       a single character in single quotes stands for the 
            Unicode code point for that character.  

2.4.2  UTF-6 Compression Algorithm

UTF-6 improves on the UTF-5 encoding by providing compression, which
enables encoding of a larger number of characters in each hostname
part.  The compression algorithm is defined as follows:

  1.  Set the mask to 0xFFFF;

  2.  If the number of non '-' characters is less than 2, proceed to 
      step 5;

  3.  If the most significant byte of every non '-' character is the
      same value:

      3a.  Set HB to this value;
      3b.  Emit 'Y';
      3c.  Emit the variable length hex encoding of HB;
      3d.  Set the mask to 0x00FF;
      3e.  Proceed to step 5.

  4.  If the most significant nibble of every non '-' character is the
      same value:

      4a.  Set HN to this value;
      4b.  Emit 'Z';
      4c.  Emit the variable length hex encoding of HN;
      4d.  Set the mask to 0x0FFF.

  5.  Foreach input character: