xcu_chap02.html

IEEE 1003.1-2003, Single Unix Specification v3

<tt>\&#152;hlj/   &#152;h\lj/   &#152;"hlj"/   &#152;hlj\/   &#152;hlj/
</tt>
</pre>
</blockquote>

<p>In an early proposal, tilde expansion occurred following any unquoted equals sign or colon, but this was removed because of its
complexity and to avoid breaking commands such as:</p>

<blockquote>
<pre>
<tt>rcp hostname:&#152;marc/.profile .
</tt>
</pre>
</blockquote>

<p>A suggestion was made that the special sequence <tt>"$&#152;"</tt> should be allowed to force tilde expansion anywhere. Since
this is not historical practice, it has been left for future implementations to evaluate. (The description in the Shell and
Utilities volume of IEEE&nbsp;Std&nbsp;1003.1-2001, <a href="../utilities/xcu_chap02.html#tag_02_02">Section 2.2, Quoting</a>
requires that a dollar sign be quoted to represent itself, so the <tt>"$&#152;"</tt> combination is already unspecified.)</p>

<p>The results of giving tilde with an unknown login name are undefined because the KornShell <tt>"&#152;+"</tt> and
<tt>"&#152;-"</tt> constructs make use of this condition, but in general it is an error to give an incorrect login name with tilde.
The results of having <i>HOME</i> unset are unspecified because some historical shells treat this as an error.</p>

<h5><a name="tag_02_02_06_02"></a>Parameter Expansion</h5>

<p>The rule for finding the closing <tt>'}'</tt> in <tt>"${...}"</tt> is the one used in the KornShell and is upwardly-compatible
with the Bourne shell, which does not determine the closing <tt>'}'</tt> until the word is expanded. The advantage of this is that
incomplete expansions, such as:</p>

<blockquote>
<pre>
<tt>${foo
</tt>
</pre>
</blockquote>

<p>can be determined during tokenization, rather than during expansion.</p>

<p>The string length and substring capabilities were included because of the demonstrated need for them, based on their usage in
other shells, such as C shell and KornShell.</p>

<p>Historical versions of the KornShell have not performed tilde expansion on the word part of parameter expansion; however, it is
more consistent to do so.</p>

<h5><a name="tag_02_02_06_03"></a>Command Substitution</h5>

<p>The <tt>"$()"</tt> form of command substitution solves a problem of inconsistent behavior when using backquotes. For
example:</p>

<center>
<table border="1" cellpadding="3" align="center">
<tr valign="top">
<th align="center">
<p class="tent"><b>Command</b></p>
</th>
<th align="center">
<p class="tent"><b>Output</b></p>
</th>
</tr>

<tr valign="top">
<td align="left">
<p class="tent">echo '\$x'</p>
</td>
<td align="left">
<p class="tent">\$x</p>
</td>
</tr>

<tr valign="top">
<td align="left">
<p class="tent">echo `echo '\$x'`</p>
</td>
<td align="left">
<p class="tent">$x</p>
</td>
</tr>

<tr valign="top">
<td align="left">
<p class="tent">echo $(echo '\$x')</p>
</td>
<td align="left">
<p class="tent">\$x</p>
</td>
</tr>
</table>
</center>

<p>Additionally, the backquoted syntax has historical restrictions on the contents of the embedded command. While the newer
<tt>"$()"</tt> form can process any kind of valid embedded script, the backquoted form cannot handle some valid scripts that
include backquotes. For example, these otherwise valid embedded scripts do not work in the left column, but do work on the
right:</p>

<blockquote>
<pre>
<tt>echo `                         echo $(
cat &lt;&lt;\eof                     cat &lt;&lt;\eof
a here-doc with `              a here-doc with )
eof                            eof
`                              )
<br>
echo `                         echo $(
echo abc # a comment with `    echo abc # a comment with )
`                              )
<br>
echo `                         echo $(
echo '`'                       echo ')'
`                              )
</tt>
</pre>
</blockquote>

<p>Because of these inconsistent behaviors, the backquoted variety of command substitution is not recommended for new applications
that nest command substitutions or attempt to embed complex scripts.</p>

<p>The KornShell feature:</p>

<blockquote>If <i>command</i> is of the form &lt; <i>word</i>, <i>word</i> is expanded to generate a pathname, and the value of the
command substitution is the contents of this file with any trailing &lt;newline&gt;s deleted.</blockquote>

<p>was omitted from the Shell and Utilities volume of IEEE&nbsp;Std&nbsp;1003.1-2001 because $( <a href=
"../utilities/cat.html"><i>cat</i></a> <i>word</i>) is an appropriate substitute. However, to prevent breaking numerous scripts
relying on this feature, it is unspecified to have a script within <tt>"$()"</tt> that has only redirections.</p>

<p>The requirement to separate <tt>"$("</tt> and <tt>'('</tt> when a single subshell is command-substituted is to avoid any
ambiguities with arithmetic expansion.</p>

<p>IEEE&nbsp;Std 1003.1-2001/Cor&nbsp;1-2002, item XCU/TC1/D6/4 is applied, changing the text from: &quot;If a command substitution
occurs inside double-quotes, it shall not be performed on the results of the substitution.&quot; to: &quot;If a command substitution occurs
inside double-quotes, field splitting and pathname expansion shall not be performed on the results of the substitution.&quot;. The
replacement text taken from the ISO&nbsp;POSIX-2:1993 standard is clearer about the items that are not performed.</p>

<h5><a name="tag_02_02_06_04"></a>Arithmetic Expansion</h5>

<p>The <tt>"(())"</tt> form of KornShell arithmetic in early proposals was omitted. The standard developers concluded that there
was a strong desire for some kind of arithmetic evaluator to replace <a href="../utilities/expr.html"><i>expr</i></a>, and that
relating it to <tt>'$'</tt> makes it work well with the standard shell language, and it provides access to arithmetic evaluation in
places where accessing a utility would be inconvenient.</p>

<p>The syntax and semantics for arithmetic were changed for the ISO/IEC&nbsp;9945-2:1993 standard. The language is essentially a
pure arithmetic evaluator of constants and operators (excluding assignment) and represents a simple subset of the previous
arithmetic language (which was derived from the KornShell <tt>"(())"</tt> construct). The syntax was changed from that of a command
denoted by ((<i>expression</i>)) to an expansion denoted by $((<i>expression</i>)). The new form is a dollar expansion (
<tt>'$'</tt> ) that evaluates the expression and substitutes the resulting value. Objections to the previous style of arithmetic
included that it was too complicated, did not fit in well with the use of variables in the shell, and its syntax conflicted with
subshells. The justification for the new syntax is that the shell is traditionally a macro language, and if a new feature is to be
added, it should be accomplished by extending the capabilities presented by the current model of the shell, rather than by
inventing a new one outside the model; adding a new dollar expansion was perceived to be the most intuitive and least destructive
way to add such a new capability.</p>

<p>In early proposals, a form $[<i>expression</i>] was used. It was functionally equivalent to the <tt>"$(())"</tt> of the current
text, but objections were lodged that the 1988 KornShell had already implemented <tt>"$(())"</tt> and there was no compelling
reason to invent yet another syntax. Furthermore, the <tt>"$[]"</tt> syntax had a minor incompatibility involving the patterns in
<b>case</b> statements.</p>

<p>The portion of the ISO&nbsp;C standard arithmetic operations selected corresponds to the operations historically supported in
the KornShell.</p>

<p>It was concluded that the <a href="../utilities/test.html"><i>test</i></a> command ( <b>[</b>) was sufficient for the majority
of relational arithmetic tests, and that tests involving complicated relational expressions within the shell are rare, yet could
still be accommodated by testing the value of <tt>"$(())"</tt> itself. For example:</p>

<blockquote>
<pre>
<tt># a complicated relational expression
while [ $(( (($x + $y)/($a * $b)) &lt; ($foo*$bar) )) -ne 0 ]
</tt>
</pre>
</blockquote>

<p>or better yet, the rare script that has many complex relational expressions could define a function like this:</p>

<blockquote>
<pre>
<tt>val() {
    return $((!$1))
}
</tt>
</pre>
</blockquote>

<p>and complicated tests would be less intimidating:</p>

<blockquote>
<pre>
<tt>while val $(( (($x + $y)/($a * $b)) &lt; ($foo*$bar) ))
do
    # some calculations
done
</tt>
</pre>
</blockquote>

<p>A suggestion that was not adopted was to modify <a href="../utilities/true.html"><i>true</i></a> and <a href=
"../utilities/false.html"><i>false</i></a> to take an optional argument, and <a href="../utilities/true.html"><i>true</i></a> would
exit true only if the argument was non-zero, and <a href="../utilities/false.html"><i>false</i></a> would exit false only if the
argument was non-zero:</p>

<blockquote>
<pre>
<tt>while true $(($x &gt; 5 &amp;&amp; $y &lt;= 25))
</tt>
</pre>
</blockquote>

<p>There is a minor portability concern with the new syntax. The example <tt>"$((2+2))"</tt> could have been intended to mean a
command substitution of a utility named <tt>"2+2"</tt> in a subshell. The standard developers considered this to be obscure and
isolated to some KornShell scripts (because <tt>"$()"</tt> command substitution existed previously only in the KornShell). The text
on command substitution requires that the <tt>"$("</tt> and <tt>'('</tt> be separate tokens if this usage is needed.</p>

<p>An example such as:</p>

<blockquote>
<pre>
<tt>echo $((echo hi);(echo there))
</tt>
</pre>
</blockquote>

<p>should not be misinterpreted by the shell as arithmetic because attempts to balance the parentheses pairs would indicate that
they are subshells. However, as indicated by the Base Definitions volume of IEEE&nbsp;Std&nbsp;1003.1-2001, <a href=
"../basedefs/xbd_chap03.html#tag_03_112">Section 3.112, Control Operator</a>, a conforming application must separate two adjacent
parentheses with white space to indicate nested subshells.</p>

<p>Although the ISO/IEC&nbsp;9899:1999 standard now requires support for <b>long long</b> and allows extended integer types with
higher ranks, IEEE&nbsp;Std&nbsp;1003.1-2001 only requires arithmetic expansions to support <b>signed long</b> integer arithmetic.
Implementations are encouraged to support signed integer values at least as large as the size of the largest file allowed on the
implementation.</p>

<p>Implementations are also allowed to perform floating-point evaluations as long as an application won't see different results for
expressions that would not overflow <b>signed long</b> integer expression evaluation. (This includes appropriate truncation of
results to integer values.)</p>

<p>Changes made in response to IEEE PASC Interpretation 1003.2 #208 removed the requirement that the integer constant suffixes
<tt>l</tt> and <tt>L</tt> had to be recognized. The ISO&nbsp;POSIX-2:1993 standard did not require the <tt>u</tt> , <tt>ul</tt> ,
<tt>uL</tt> , <tt>U</tt> , <tt>Ul</tt> , <tt>UL</tt> , <tt>lu</tt> , <tt>lU</tt> , <tt>Lu</tt> , and <tt>LU</tt> suffixes since
only signed integer arithmetic was required. Since all arithmetic expressions were treated as handling <b>signed long</b> integer
types anyway, the <tt>l</tt> and <tt>L</tt> suffixes were redundant. No known scripts used them and some historic shells did not
support them. When the ISO/IEC&nbsp;9899:1999 standard was used as the basis for the description of arithmetic processing, the
<tt>ll</tt> and <tt>LL</tt> suffixes and combinations were also not required. Implementations are still free to accept any or all
of these suffixes, but are not required to do so.</p>

<p>There was also some confusion as to whether the shell was required to recognize character constants. Syntactically, character
constants were required to be recognized, but the requirements for the handling of backslash ( <tt>'\'</tt> ) and quote (
<tt>'&quot;</tt> ) characters (needed to specify character constants) within an arithmetic expansion were ambiguous. Furthermore, no
known shells supported them. Changes made in response to IEEE PASC Interpretation 1003.2 #208 removed the requirement to support
them (if they were indeed required before). IEEE&nbsp;Std&nbsp;1003.1-2001 clearly does not require support for character
constants.</p>

<h5><a name="tag_02_02_06_05"></a>Field Splitting</h5>

<p>The operation of field splitting using <i>IFS ,</i> as described in early proposals, was based on the way the KornShell splits
words, but it is incompatible with other common versions of the shell. However, each has merit, and so a decision was made to allow
both. If the <i>IFS</i> variable is unset or is &lt;space&gt; &lt;tab&gt; &lt;newline&gt;, the operation is equivalent to the way
the System&nbsp;V shell splits words. Using characters outside the &lt;space&gt; &lt;tab&gt; &lt;newline&gt; set yields the
KornShell behavior, where each of the non- &lt;space&gt; &lt;tab&gt; &lt;newline&gt;s is significant. This behavior, which affords
the most flexibility, was taken from the way the original <a href="../utilities/awk.html"><i>awk</i></a> handled field
splitting.</p>

<p>Rule (3) can be summarized as a pseudo-ERE:</p>

<blockquote>
<pre>
<tt>(</tt><i>s</i><tt>*</tt><i>ns</i><tt>*|</tt><i>s</i><tt>+)
</tt>
</pre>
</blockquote>

<p>where <i>s</i> is an <i>IFS</i> white space character and <i>n</i> is a character in the <i>IFS</i> that is not white space. Any
string matching that ERE delimits a field, except that the <i>s</i>+ form does not delimit fields at the beginning or the end of a
line. For example, if <i>IFS</i> is &lt;space&gt;/ &lt;comma&gt;/ &lt;tab&gt;, the string:</p>

<blockquote>
<pre>
<tt>&lt;space&gt;&lt;space&gt;red&lt;space&gt;&lt;space&gt;,&lt;space&gt;white&lt;space&gt;blue
</tt>
</pre>
</blockquote>

<p>yields the three colors as the delimited fields.</p>

<h5><a name="tag_02_02_06_06"></a>Pathname Expansion</h5>

<p>There is no additional rationale provided for this section.</p>

<h5><a name="tag_02_02_06_07"></a>Quote Removal</h5>

<p>There is no additional rationale provided for this section.</p>

<h4><a name="tag_02_02_07"></a>Redirection</h4>

<p>In the System Interfaces volume of IEEE&nbsp;Std&nbsp;1003.1-2001, file descriptors are integers in the range 0-({OPEN_MAX}-1).
The file descriptors discussed in the Shell and Utilities volume of IEEE&nbsp;Std&nbsp;1003.1-2001, <a href=
"../utilities/xcu_chap02.html#tag_02_07">Section 2.7, Redirection</a> are that same set of small integers.</p>

<p>Having multi-digit file descriptor numbers for I/O redirection can cause some obscure compatibility problems. Specifically,
scripts that depend on an example command:</p>

<blockquote>
<pre>
<tt>echo 22&gt;/dev/null
</tt>
</pre>
</blockquote>