2

I have the following command

echo '123456789#' | xargs -I% bash -c 'echo %%%%%%%%%%%%%%%%%%%%%%%%%'

It doesn't expand the last %. Apparently there's a limit on how long the string passed to bash -c can be. Or maybe it is size of the command that can be passed to xargs? I am curious what kind of constraint is responsible for this behaviour and can it be configured?

Running

echo 'x' | xargs -I% bash -c 'echo %%%%%%%%%%%%%%%%%%%%%%%%%'

gives xxxxxxxxxxxxxxxxxxxxxxxxx so all 25 occurences of % are expanded, which means it's not the number of occurences of %, but rather length of the resulting string

Note: I run this command on MacOS Ventura 13.6.1

• bash
• macos
• xargs

Share

Improve this question

Follow

edited Jul 2, 2024 at 7:58

asked Jul 1, 2024 at 12:25

 

gicig

12144 bronze badges

• 1

  See what-defines-the-maximum-size-for-a-command-single-argument but FWIW I see that string repeated 25 times and less tan 250 chars would be very small for a ARG_MAX setting so you may have something else going on. What happens if you use echo 'x' | xargs ... instead? Do you get 25 xs or not? From testing with other input strings, is the output truncated at 240 chars or 24 repetitions of the input or something else? Or do you see a literal % at the end of the output? 

  – Ed Morton

   CommentedJul 1, 2024 at 12:36 

• 1

  In general a good practice is not to embed your % in the shell code in the first place. echo '123456789#' | xargs -I% bash -c 'echo "$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1"' bash % 

  – Kamil Maciorowski

   CommentedJul 1, 2024 at 12:42
• @EdMorton I get 25 x 

  – gicig

   CommentedJul 1, 2024 at 12:49

• 1

  Please edit your question to provide the answers to all of my questions. Don't add information in comments where it can't be formatted and could be missed. Also add the exact output you get from the script you posted. Thanks. 

  – Ed Morton

   CommentedJul 1, 2024 at 12:53 

• 1

  A portable xargs shall follow this: "At least five arguments in arguments can each contain one or more instances of replstr. Each of these constructed arguments cannot grow larger than an implementation-defined limit greater than or equal to 255 bytes". So maybe it's your xargs that is limited; which is another reason to use the code from my first comment where % gets expanded exactly once and the resulting string is short; then the long string to echo is built in bash, not by xargs. 

  – Kamil Maciorowski

   CommentedJul 1, 2024 at 13:02 

Show 3 more comments

1 Answer

Sorted by:

                                              Highest score (default)                                                                   Date modified (newest first)                                                                   Date created (oldest first)                              

1

Linux has a limit of 128 kB for the length of a single argument string, but that's a lot longer a limit than the string you tried. As far as I understand, most other systems have no specific limit for the length of a single argument, but of course have some limits for their total length (it'll have to be limited by available memory anyway, possibly stack space in particular). If Bash itself has a limit, it's likely long enough for all practical purposes.

Se e.g. this should work:

% bash -c "echo $(printf "1234567890%.0s" {0..30}) | rev"
0987654321098765432109876543210987654321098765432109876543210987654321098765432109876543210987654321098765432109876543210987654321098765432109876543210987654321098765432109876543210987654321098765432109876543210987654321098765432109876543210987654321098765432109876543210987654321098765432109876543210987654321

The fact that your command leaves some of the %s unexpanded and that it depends on the resulting length, hints at that being an issue of that xargs implementation. (Is it the GNU one? I'm slightly surprised if they have such a low limit.)

---

In any case, as mentioned in the comments, it's not such a good idea to embed data directly into your shell script as here. It can lead to issues with strange inputs (think quotes), and those can escalate into security problems. Best to avoid doing it.

Instead, pass the data string as an argument to Bash, where it will be available as $1 in the script (the following args being $2, $3 etc).

echo '123456789#' | xargs -I{} bash -c 'echo "$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1$1"' bash {}

(Note that you still can't pass strings with quotes or backslashes (or that start with whitespace) into that without further consideration, since xargs itself does some quote processing on the input unless you use non-standard options such as the -0 / -d of GNU xargs.)

Share

Improve this answer

Follow

edited Jul 3, 2024 at 9:46

community wiki

3 revs, 2 users 96%
ilkkachu

• I forgot to mention (added that in the question) that I was running that command on MacOS Ventura 13.6.1. I guess there is difference between that implemention and one available on Linux 

  – gicig

   CommentedJul 2, 2024 at 9:02
• @gicig, yes, macos has some (possibly old?) version of the FreeBSD toolset, or something like that. They don't support GNU extensions anyway. You just had the question tagged with linux at first, so I guessed at GNU 

  – ilkkachu

   CommentedJul 2, 2024 at 13:18