Grabbing the first [x] characters for a string from a pipe

Question

If I have really long output from a command (single line) but I know I only want the first [x] (let's say 8) characters of the output, what's the easiest way to get that? There aren't any delimiters.

Answer 1

One way is to use cut:

 command | cut -c1-8

This will give you the first 8 characters of each line of output. Since cut is part of POSIX, it is likely to be on most Unices.

Answer 2

These are some other ways to get only first 8 characters.

command | head -c8

command | awk '{print substr($0,1,8);exit}' 

command | sed 's/^\(........\).*/\1/;q'

And if you have bash

var=$(command)
echo ${var:0:8}

Answer 3

Another one liner solution by using Shell parameter expansion

echo ${word:0:x}

EG: word="Hello world"
echo ${word:0:3} or echo ${word::3} 
o/p: Hel


EG.2: word="Hello world"
echo ${word:1:3}
o/p: ell

Answer 4

If you have a sufficiently advanced shell (for example, the following will work in Bash, not sure about dash), you can do:

read -n8 -d$'\0' -r <(command)

After executing read ... <(command), your characters will be in the shell variable REPLY. Type help read to learn about other options.

Explanation: the -n8 argument to read says that we want up to 8 characters. The -d$'\0' says read until a null, rather than to a newline. This way the read will continue for 8 characters even if one of the earlier characters is a newline (but not if its a null). An alternative to -n8 -d$'\0' is to use -N8, which reads for exactly 8 characters or until the stdin reaches EOF. No delimiter is honored. That probably fits your needs better, but I don't know offhand how many shells have a read that honors -N as opposed to honoring -n and -d. Continuing with the explanation: -r says ignore \-escapes, so that, for example, we treat \\ as two characters, rather than as a single \.

Finally, we do read ... <(command) rather than command | read ... because in the second form, the read is executed in a subshell which is then immediately exited, losing the information you just read.

Another option is to do all your processing inside the subshell. For example:

$ echo abcdefghijklm | { read -n8 -d$'\0' -r; printf "REPLY=<%s>\n" "$REPLY"; }
REPLY=<abcdefgh>

Answer 5

This is portable:

a="$(command)"             # Get the output of the command.
b="????"                   # as many ? as characters are needed.
echo ${a%"${a#${b}}"}      # select that many chars from $a

To build a string of variable length of characters has its own question here.

Answer 6

I had this problem when manually generating checksum files in maven repository. Unfortunately cut -c always prints out a newline at the end of output. To suppress that I use xxd:

command | xxd -l$BYTES | xxd -r

It outputs exactly $BYTES bytes, unless the command's output is shorter, then exactly that output.

Answer 7

How to consider Unicode + UTF-8

Let's do a quick test for those interested in Unicode characters rather than just bytes. Each character of áéíóú (acute accented vowels) is made up of two bytes in UTF-8. With:

printf 'áéíóú' | LC_CTYPE=en_US.UTF-8 awk '{print substr($0,1,3);exit}'
printf 'áéíóú' | LC_CTYPE=C awk '{print substr($0,1,3);exit}'
printf 'áéíóú' | LC_CTYPE=en_US.UTF-8 head -c3
echo
printf 'áéíóú' | LC_CTYPE=C head -c3
echo

we get:

áéí
á
á
á

so we see that only awk + LC_CTYPE=en_US.UTF-8 considered the UTF-8 characters. The other approaches took only three bytes. We can confirm that with:

printf 'áéíóú' | LC_CTYPE=C head -c3 | hd

which gives:

00000000  c3 a1 c3                                          |...|
00000003

and the c3 by itself is trash, and does not show up on the terminal, so we saw only á.

awk + LC_CTYPE=en_US.UTF-8 actually returns 6 bytes however.

We could also have equivalently tested with:

printf '\xc3\xa1\xc3\xa9\xc3\xad\xc3\xb3\xc3\xba' | LC_CTYPE=en_US.UTF-8 awk '{print substr($0,1,3);exit}'

and if you want a general parameter:

n=3
printf 'áéíóú' | LC_CTYPE=en_US.UTF-8 awk "{print substr(\$0,1,$n);exit}"

Question more specific about Unicode + UTF-8: https://superuser.com/questions/450303/unix-tool-to-output-first-n-characters-in-an-utf-8-encoded-file

Related: https://stackoverflow.com/questions/1405611/how-to-extract-the-first-two-characters-of-a-string-in-shell-scripting

Tested on Ubuntu 21.04.

Answer 8

With zsh, you can do:

cmd | read -u0 -k4 -e

read will read as many bytes as needed to read 4 characters (k was initially for key, but with -u specifying a file descriptor it reads characters from there instead of key presses from the terminal) and echo (-e) those characters on stdout. You can change -e to a variable name to read those bytes into a variable.

ksh93 later (in ksh93o in 2003, while zsh's -k is from the 90s) added a -N options as an equivalent of zsh's -k which was later copied by bash (though with some differences, see below). It doesn't have an equivalent for -e though.

cmd | read -N4 var

Contrary to zsh, ksh93 cannot store NUL characters in its variables, and more generally, it will fail in random ways if there are NULs in its input.

Now, besides -N, ksh93 also has a -n x option which reads up to x characters from a line, and the record delimiter can be changed with -d, and with recent versions of ksh93u+m, -d '' is for NUL-delimited records.

So:

cmd | read -d '' -n 4

there fails in a less random way if the input contains NUL characters: it just stops at the first NUL.

Now, bash copied all of -n, -N, -d (including -d '') from ksh93 but with important differences:

• it still does backslash processing when -n/-N are specified, so you need -r to work around it as usual.
• it still does IFS processing, which you need to work around by calling it as IFS= read... as usual
• it skips all NUL characters in its input
• by default, the last component of a pipeline also runs in a subshell, which you can work around by using a redirection to a process substitution.

So in bash, you'd do:

IFS= read -rN4 var < <(cmd)

To read the first non-NUL characters of the output of cmd or (if cmd is cat /dev/zero, it will never return). And:

IFS= read -d '' -rn4 var < <(cmd)

To read 4 characters up to the first NUL.