Commit Graph

2192 Commits

Author SHA1 Message Date
Denys Vlasenko
d5f5045b43 ash: expand: Fix buffer overflow in expandmeta
Upstream commit:

    Date: Sun, 25 Mar 2018 16:38:00 +0800
    expand: Fix buffer overflow in expandmeta

    The native version of expandmeta allocates a buffer that may be
    overrun for two reasons.  First of all the size is 1 byte too small
    but this is normally hidden because the minimum size is rounded
    up to 2048 bytes.  Secondly, if the directory level is deep enough,
    any buffer can be overrun.

    This patch fixes both problems by calling realloc when necessary.

    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

function                                             old     new   delta
expmeta                                              517     635    +118
expandarg                                            990     996      +6
mklocal                                              288     290      +2
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 3/0 up/down: 126/0)             Total: 126 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-14 14:50:47 +02:00
Denys Vlasenko
46158dc833 shell: add 6856 $IFS tests to testsuites
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 20:24:58 +02:00
Denys Vlasenko
f693b606b7 hush: fix recent breakage from parse_stream() changes
function                                             old     new   delta
parse_stream                                        3808    3821     +13

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 20:00:43 +02:00
Denys Vlasenko
44257ad5d0 hush: fix IFS handling in read
$ echo "X:Y:" | (IFS=": " read x y; echo "|$x|$y|")
|X|Y|
$ echo "X:Y  :  " | (IFS=": " read x y; echo "|$x|$y|")
|X|Y|

function                                             old     new   delta
shell_builtin_read                                  1320    1426    +106

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 17:18:34 +02:00
Denys Vlasenko
9678636911 hush: IFS fixes
$ IFS=": "; x=" "; set x $x; for v; do echo "|$v|"; done
|x|
$ IFS=": "; x=":"; set x $x; for v; do echo "|$v|"; done
|x|
||

function                                             old     new   delta
run_pipe                                            1789    1870     +81
expand_on_ifs                                        310     361     +51
pseudo_exec_argv                                     588     591      +3
builtin_local                                         50      53      +3
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 4/0 up/down: 138/0)             Total: 138 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 16:02:58 +02:00
Denys Vlasenko
34179956f9 hush: fix "$v" expansion in case patterns when v='[a]'
function                                             old     new   delta
run_list                                            1053    1063     +10
setup_redirects                                      311     320      +9
encode_then_expand_string                            135     142      +7
run_pipe                                            1784    1789      +5
expand_assignments                                    81      86      +5
expand_string_to_string                              124     125      +1
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 6/0 up/down: 37/0)               Total: 37 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 13:47:59 +02:00
Denys Vlasenko
680c3016a2 ash: parser: Allow newlines within parameter substitution
Upstream commit:

Date: Thu, 22 Mar 2018 21:41:24 +0800
parser: Allow newlines within parameter substitution

    On Fri, Mar 16, 2018 at 11:27:22AM +0800, Herbert Xu wrote:
    > On Thu, Mar 15, 2018 at 10:49:15PM +0100, Harald van Dijk wrote:
    > >
    > > Okay, it can be trivially modified to something that does work in other
    > > shells (even if it were actually executed), but gets rejected at parse time
    > > by dash:
    > >
    > >   if false; then
    > >     : ${$+
    > >   }
    > >   fi
    >
    > That's just a bug in dash's parser with ${} in general, because
    > it bombs out without the if clause too:
    >
    > 	: ${$+
    > 	}

    This patch fixes the parsing of newlines with parameter substitution.

    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 12:39:18 +02:00
Denys Vlasenko
0403bedccc hush: optimize parse_stream()
Since we check for '\' anyway when we determine whether we can look ahead,
we can just check for *and handle* it there.

function                                             old     new   delta
parse_stream                                        2751    2740     -11

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 01:34:46 +02:00
Denys Vlasenko
89e9d5534d hush: do not drop backslash from eval 'echo ok\'
newer bash does not drop it, most other shells too

function                                             old     new   delta
unbackslash                                           39      57     +18
parse_stream                                        2753    2751      -2
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/1 up/down: 18/-2)              Total: 16 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-11 01:15:33 +02:00
Denys Vlasenko
3632cb15f1 shell: add comments about [[, no code changes
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 15:25:41 +02:00
Denys Vlasenko
4709df0f15 hush: fix handling of \<eof> in double-quoted strings
function                                             old     new   delta
encode_string                                        268     250     -18

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 14:49:01 +02:00
Denys Vlasenko
bcf56114fa hush: fix eval 'echo ok\'
function                                             old     new   delta
parse_stream                                        2762    2753      -9

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 14:40:23 +02:00
Denys Vlasenko
92a930b4e8 hush: simplify \<newline> code, part 3
function                                             old     new   delta
parse_stream                                        2780    2762     -18

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 14:27:08 +02:00
Denys Vlasenko
e8b1bc0481 hush: simplify \<newline> code, part 2
function                                             old     new   delta
parse_stream                                        2787    2780      -7

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 14:23:49 +02:00
Denys Vlasenko
1c57269b5d hush: simplify \<newline> code, part 1
function                                             old     new   delta
parse_stream                                        2919    2787    -132

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 14:23:49 +02:00
Denys Vlasenko
09b7a7ec0e hush: put "current word" structure into parsing context
function                                             old     new   delta
done_word                                            790     767     -23
parse_stream                                        3018    2919     -99
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 0/2 up/down: 0/-122)           Total: -122 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 03:22:10 +02:00
Denys Vlasenko
e93031e6dc ash: if "[[" bashism is not supported, do not handle it anywhere
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 01:23:19 +02:00
Denys Vlasenko
57b7efb0d5 ash: trivial code shrink
function                                             old     new   delta
parse_command                                       1677    1674      -3

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-10 01:20:26 +02:00
Denys Vlasenko
9db344a0f4 hush: fix var_leaks.tests and var_preserved.tests on NOMMU
function                                             old     new   delta
remove_nested_vars                                     -      77     +77
run_pipe                                            1756    1786     +30
pseudo_exec_argv                                     376     379      +3
leave_var_nest_level                                  98      32     -66
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 2/1 up/down: 110/-66)            Total: 44 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-09 19:05:11 +02:00
Denys Vlasenko
eb0de05d68 hush: fix func_return2.tests on NOMMU
function                                             old     new   delta
hush_main                                           1714    1718      +4

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-09 17:54:07 +02:00
Denys Vlasenko
38ccd6af8a bzip2: fix two crashes on corrupted archives
As it turns out, longjmp'ing into freed stack is not healthy...

function                                             old     new   delta
unpack_usage_messages                                  -      97     +97
unpack_bz2_stream                                    369     409     +40
get_next_block                                      1667    1677     +10
get_bits                                             156     155      -1
start_bunzip                                         212     183     -29
bb_show_usage                                        181     120     -61
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 2/3 up/down: 147/-91)            Total: 56 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-08 20:05:04 +02:00
Denys Vlasenko
7bcde5f00d libbb.h: always include sys/resource.h
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-08 17:23:27 +02:00
Denys Vlasenko
8d6eab3225 hush: fix prompt in multi-line $(())
Now shows PS2 in this case:

/path/to/dir $ a=b; echo $((
> _

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-07 17:01:31 +02:00
Denys Vlasenko
f5018dac21 hush: fix "unset PS1/PS2", and put them into initial variable set
"unset PS1/PS2" causes prompts to be empty strings

function                                             old     new   delta
hush_main                                           1031    1089     +58
goto_new_line                                         27      33      +6
fgetc_interactive                                    244     245      +1
unset_local_var                                      155     149      -6
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 3/1 up/down: 65/-6)              Total: 59 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-06 17:58:21 +02:00
Denys Vlasenko
00bd76728d hush: if we did match "LINENO" or "OPTIND", stop further comparisons
function                                             old     new   delta
handle_changed_special_names                          99     101      +2

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-06 14:57:53 +02:00
Denys Vlasenko
cf079ffe1c hush: consolidate handling of setting/unsetting of PSn, LINENO, OPTIND
function                                             old     new   delta
handle_changed_special_names                           -      99     +99
unset_local_var                                      256     155    -101
set_local_var                                        557     437    -120
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 0/2 up/down: 99/-221)          Total: -122 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-06 14:50:12 +02:00
Denys Vlasenko
de02625985 hush: update to correctly handle changed var_bash[346].tests
function                                             old     new   delta
expand_one_var                                      1612    1604      -8

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 17:04:53 +02:00
Denys Vlasenko
f2ed39b930 hush: implement "hush -s"
function                                             old     new   delta
hush_main                                           1015    1031     +16
packed_usage                                       32757   32745     -12
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/1 up/down: 16/-12)              Total: 4 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 16:46:49 +02:00
Denys Vlasenko
21b7f1b6b6 hush: fix a few more corner cases with empty-expanding cmds
See added testcases

function                                             old     new   delta
run_pipe                                            1723    1784     +61

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 15:15:53 +02:00
Denys Vlasenko
41d8f10813 hush: fix corner cases with exec in empty expansions
Cases like these:

var=val exec >redir

var=val `` >redir

function                                             old     new   delta
run_pipe                                            1701    1723     +22
redirect_and_varexp_helper                            56      55      -1
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 1/1 up/down: 22/-1)              Total: 21 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 14:41:21 +02:00
Denys Vlasenko
929a41d577 hush: less mind-bending set_vars_and_save_old()
function                                             old     new   delta
run_pipe                                            1651    1701     +50
set_local_var                                        510     557     +47
pseudo_exec_argv                                     544     581     +37
redirect_and_varexp_helper                            64      56      -8
set_vars_and_save_old                                164     149     -15
unset_local_var                                      274     256     -18
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 3/3 up/down: 134/-41)            Total: 93 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 14:09:14 +02:00
Denys Vlasenko
4e1dc539e9 hush: "no logic changes" in last commit was not true, fix it up
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 13:10:34 +02:00
Denys Vlasenko
34f6b12330 hush: make run_pipe code simpler to understand, no logic changes
function                                             old     new   delta
run_pipe                                            1641    1651     +10

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 11:30:17 +02:00
Denys Vlasenko
d358b0b65d hush: fix a bug where we don't properly handle f() { a=A; b=B; }; a= f
function                                             old     new   delta
unset_local_var                                       20     274    +254
leave_var_nest_level                                   -      98     +98
set_vars_and_save_old                                128     164     +36
enter_var_nest_level                                   -      32     +32
builtin_local                                         46      50      +4
pseudo_exec_argv                                     554     544     -10
redirect_and_varexp_helper                            77      64     -13
run_pipe                                            1890    1641    -249
unset_local_var_len                                  267       -    -267
------------------------------------------------------------------------------
(add/remove: 2/1 grow/shrink: 3/3 up/down: 424/-539)         Total: -115 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-05 00:51:55 +02:00
Denys Vlasenko
332e4115c9 hush: make var nesting code independent of "local" support
Also, add code to abort at ~65000 function recursion depth.
SEGVing is not as nice as exiting with a message (and restoring termios!):

$ f() { echo -n .; f; }; f
....<many dots later>....hush: fatal recursion (depth 65281)

function                                             old     new   delta
run_pipe                                            1826    1890     +64
pseudo_exec_argv                                     544     554     +10
parse_and_run_file                                    71      80      +9
i_getch                                              104     107      +3
done_command                                          99     102      +3
set_local_var                                        508     510      +2
helper_export_local                                  214     215      +1
builtin_local                                         49      46      -3
------------------------------------------------------------------------------
(add/remove: 0/0 grow/shrink: 7/1 up/down: 92/-3)              Total: 89 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-04 22:35:13 +02:00
Denys Vlasenko
61407807ab hush: fix for readonly vars in "ro=A ro=B cmd" case
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-04 21:14:28 +02:00
Denys Vlasenko
ee1fd1246e ash: unbreak PS1 parsing after "ash: parser: Add syntax stack..." commit
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-04 13:59:53 +02:00
Denys Vlasenko
fbf44854a3 hush: support "f() (cmd)" functions
Many other shells support this construct

function                                             old     new   delta
parse_stream                                        2950    3018     +68

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-03 14:56:52 +02:00
Denys Vlasenko
49015a60cb hush: fix mishandling of "true | f() { echo QWE; }"
function                                             old     new   delta
run_pipe                                            1820    1826      +6

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-03 13:02:43 +02:00
Denys Vlasenko
5fa0505f8a hush: fix "set -e; false || x=1; echo OK"
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-03 11:21:13 +02:00
Denys Vlasenko
11752d46d1 hush: one-word, no-globbing handling of local/export/readonly args
function                                             old     new   delta
done_word                                            738     790     +52

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-03 08:20:58 +02:00
Denys Vlasenko
f50e14632f ash: parser: Fix parameter expansion inside inner double quotes
Upstream email:

    parser: Fix parameter expansion inside inner double quotes

    The parsing of parameter expansion inside inner double quotes
    breaks because we never look for ENDVAR while innerdq is true.

            echo "${x#"${x+''}"''}

    This patch fixes it by pushing the syntax stack if innerdq is
    true and we enter a new parameter expansion.

    This patch also fixes a corner case where a bad substitution error
    occurs within arithmetic expansion.

    Reported-by: Denys Vlasenko <vda.linux@googlemail.com>
    Fixes: ab1cecb40478 (" parser: Add syntax stack for recursive...")
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

function                                             old     new   delta
readtoken1                                          2880    2898     +18

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 21:00:59 +02:00
Denys Vlasenko
abf755615e hush: fix a backslash-removal bug in case
function                                             old     new   delta
run_list                                            1270    1053    -217

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 17:25:18 +02:00
Denys Vlasenko
9a95df9046 ash: expand: Fix bugs with words connected to the right of $@
Upstream email:

    This is actually composed of two bugs.  First of all our tracking
    of quotemark is wrong so anything after "$@" becomes quoted.  Once
    we fix that then the problem is that the first space character
    after "$@" is not recognised as an IFS.

    This patch fixes both.

    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 14:27:50 +02:00
Denys Vlasenko
355ec353be ash: redir: Fix typo in noclobber code
Upstream commit "redir: Fix typo in noclobber code"

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 13:34:57 +02:00
Denys Vlasenko
c4c2012284 ash: parser: Fix single-quoted patterns in here-documents
Upstream commit:

    From: Herbert Xu <herbert@gondor.apana.org.au>
    Date: Fri, 9 Mar 2018 23:07:53 +0800
    parser: Fix single-quoted patterns in here-documents

    The script

            x=*
            cat <<- EOF
                    ${x#'*'}
            EOF

    prints * instead of nothing as it should.  The problem is that
    when we're in sqsyntax context in a here-document, we won't add
    CTLESC as we should.  This patch fixes it:

    Reported-by: Harald van Dijk <harald@gigawatt.nl>
    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 13:29:20 +02:00
Denys Vlasenko
8b536eb40d hush: remove stray debugging printout
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 13:26:16 +02:00
Denys Vlasenko
216913c290 ash: parser: Add syntax stack for recursive parsing
This closes 10821.

Upstream patch:

    From: Herbert Xu <herbert@gondor.apana.org.au>
    Date: Fri, 9 Mar 2018 00:14:02 +0800
    parser: Add syntax stack for recursive parsing

    Without a stack of syntaxes we cannot correctly these two cases
    together:

            "${a#'$$'}"
            "${a#"${b-'$$'}"}"

    A recursive parser also helps in some other corner cases such
    as nested arithmetic expansion with paratheses.

    This patch adds a syntax stack allocated from the stack using
    alloca.  As a side-effect this allows us to remove the naked
    backslashes for patterns within double-quotes, which means that
    EXP_QPAT also has to go.

    This patch also fixes removes any backslashes that precede right
    braces when they are present within a parameter expansion context,
    and backslashes that precede double quotes within inner double
    quotes inside a parameter expansion in a here-document context.

    The idea of a recursive parser is based on a patch by Harald van
    Dijk.

    Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

var_bash3, var_bash4 and var_bash6 tests are updated
with the output given by bash-4.3.43

With this patch, the following tests now pass for ash:

    dollar_repl_slash_bash2.tests
    squote_in_varexp2.tests
    squote_in_varexp.tests
    var_bash4.tests

function                                             old     new   delta
readtoken1                                          2615    2874    +259
synstack_push                                          -      54     +54
evalvar                                              574     571      -3
rmescapes                                            330     310     -20
subevalvar                                          1279    1258     -21
argstr                                              1146    1107     -39
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 1/4 up/down: 313/-83)           Total: 230 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-02 13:15:37 +02:00
Denys Vlasenko
e84212f834 hush: update information comment about heredoc discrepancy
Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-01 20:11:23 +02:00
Denys Vlasenko
899ae5337a libbb: new function bb_die_memory_exhausted
function                                             old     new   delta
bb_die_memory_exhausted                                -      10     +10
xstrdup                                               28      23      -5
xsetenv                                               27      22      -5
xrealloc                                              32      27      -5
xputenv                                               22      17      -5
xmalloc                                               30      25      -5
xfdopen_helper                                        40      35      -5
xasprintf                                             44      39      -5
wget_main                                           2387    2382      -5
open_socket                                           54      49      -5
glob_brace                                           419     414      -5
bb_get_chunk_from_file                               146     141      -5
------------------------------------------------------------------------------
(add/remove: 1/0 grow/shrink: 0/11 up/down: 10/-55)           Total: -45 bytes

Signed-off-by: Denys Vlasenko <vda.linux@googlemail.com>
2018-04-01 19:59:37 +02:00