busybox/shell
Denys Vlasenko 216913c290 ash: parser: Add syntax stack for recursive parsing
This closes 10821.

Upstream patch:

    From: Herbert Xu <herbert@gondor.apana.org.au>
    Date: Fri, 9 Mar 2018 00:14:02 +0800
    parser: Add syntax stack for recursive parsing

    Without a stack of syntaxes we cannot correctly these two cases
    together:

            "${a#'$$'}"
            "${a#"${b-'$$'}"}"

    A recursive parser also helps in some other corner cases such
    as nested arithmetic expansion with paratheses.

    This patch adds a syntax stack allocated from the stack using
    alloca.  As a side-effect this allows us to remove the naked
    backslashes for patterns within double-quotes, which means that
    EXP_QPAT also has to go.

    This patch also fixes removes any backslashes that precede right
    braces when they are present within a parameter expansion context,
    and backslashes that precede double quotes within inner double
    quotes inside a parameter expansion in a here-document context.

    The idea of a recursive parser is based on a patch by Harald van
    Dijk.

    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

var_bash3, var_bash4 and var_bash6 tests are updated
with the output given by bash-4.3.43

With this patch, the following tests now pass for ash:

    dollar_repl_slash_bash2.tests
    squote_in_varexp2.tests
    squote_in_varexp.tests
    var_bash4.tests

function                                             old     new   delta
readtoken1                                          2615    2874    +259
synstack_push                                          -      54     +54
evalvar                                              574     571      -3
rmescapes                                            330     310     -20
subevalvar                                          1279    1258     -21
argstr                                              1146    1107     -39
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 1/4 up/down: 313/-83)           Total: 230 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 13:15:37 +02:00
..
ash_test ash: parser: Add syntax stack for recursive parsing 2018-04-02 13:15:37 +02:00
hush_test ash: parser: Add syntax stack for recursive parsing 2018-04-02 13:15:37 +02:00
ash_doc.txt
ash_ptr_hack.c *: make GNU licensing statement forms more regular 2010-08-16 20:14:46 +02:00
ash.c ash: parser: Add syntax stack for recursive parsing 2018-04-02 13:15:37 +02:00
brace.txt hush: wait for cmd to complete, and immediately store its exitcode in $? 2009-11-15 19:58:19 +01:00
Config.src config: deindent all help texts 2017-07-21 09:50:55 +02:00
cttyhack.c regularize format of source file headers, no code changes 2017-09-18 16:28:43 +02:00
hush_doc.txt
hush_leaktool.sh hush: fix "export not_yet_defined_var", fix parsing of "cmd | }" 2009-04-19 23:07:51 +00:00
hush.c hush: update information comment about heredoc discrepancy 2018-04-01 20:11:23 +02:00
Kbuild.src Make it possible to select "sh" and "bash" aliases without selecting ash or hush 2016-12-23 16:56:43 +01:00
match.c hush: fix a='a\\'; echo "${a%\\\\}" 2018-03-02 20:48:36 +01:00
match.h hush: optimize #[#] and %[%] for speed. size -2 bytes. 2010-09-04 21:21:07 +02:00
math.c shell: handle $((NUM++...) like bash does. Closes 10706 2018-01-28 20:13:33 +01:00
math.h Make it possible to select "sh" and "bash" aliases without selecting ash or hush 2016-12-23 16:56:43 +01:00
random.c ash,hush: fix a thinko about 2^64-1 factorization 2014-03-15 09:25:46 +01:00
random.h ash,hush: improve randomness of $RANDOM, add easy-ish way to test it 2014-03-13 12:52:43 +01:00
README update shell/README 2010-05-20 12:56:14 +02:00
README.job
shell_common.c hush: GETOPT_RESET() _after_ getopts too. 2017-08-29 14:32:17 +02:00
shell_common.h shell: more efficient check for EOL in read 2017-08-09 14:04:07 +02:00

http://www.opengroup.org/onlinepubs/9699919799/
Open Group Base Specifications Issue 7


http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap01.html
Shell & Utilities

It says that any of the standard utilities may be implemented
as a regular shell built-in. It gives a list of utilities which
are usually implemented that way (and some of them can only
be implemented as built-ins, like "alias"):

alias
bg
cd
command
false
fc
fg
getopts
jobs
kill
newgrp
pwd
read
true
umask
unalias
wait


http://www.opengroup.org/onlinepubs/9699919799/utilities/V3_chap02.html
Shell Command Language

It says that shell must implement special built-ins. Special built-ins
differ from regular ones by the fact that variable assignments
done on special builtin are *PRESERVED*. That is,

VAR=VAL special_builtin; echo $VAR

should print VAL.

(Another distinction is that an error in special built-in should
abort the shell, but this is not such a critical difference,
and moreover, at least bash's "set" does not follow this rule,
which is even codified in autoconf configure logic now...)

List of special builtins:

. file
: [argument...]
break [n]
continue [n]
eval [argument...]
exec [command [argument...]]
exit [n]
export name[=word]...
export -p
readonly name[=word]...
readonly -p
return [n]
set [-abCefhmnuvx] [-o option] [argument...]
set [+abCefhmnuvx] [+o option] [argument...]
set -- [argument...]
set -o
set +o
shift [n]
times
trap n [condition...]
trap [action condition...]
unset [-fv] name...

In practice, no one uses this obscure feature - none of these builtins
gives any special reasons to play such dirty tricks.

However. This section also says that *function invocation* should act
similar to special built-in. That is, variable assignments
done on function invocation should be preserved after function invocation.

This is significant: it is not unthinkable to want to run a function
with some variables set to special values. But because of the above,
it does not work: variable will "leak" out of the function.