Use sed command with variables

Question

I have tried every possible solution available on the Stack Overflow and its related websites but did not find any solution. I spent a reasonable amount of time on this issue and am finally posting this question.

I want to use the sed command with shell variables. My script is pretty simple:

## string to replace is the text after comma in the variable pc.
to_replace=${pc#*,}
echo $to_replace

##text to be replaced by the following line
replace_with="PARTITIONED BY ($pc1);"
echo $replace_with

## use sed command to replace.
sed "s@$to_replace@$replace_with@" $entry  ## $entry is the variable that contains the file name

The two echo commands give the following outputs respectively:

PARTITIONEDED BY (date_key );  ## the text I want to be replaced
PARTITIONED BY ( date_key int );  ## the text I want to replace with

I either get an error:

sed: -e expression #1, char 2: unterminated `s' command

or the text does not get replaced at all.

Someone please help. I am using Centos 6 (if that matters). Thanks in advance!

Answer 1

sed: -e expression #1, char 2: unterminated `s' command

The error message says the error happens on the second character, which seems curious. I can reproduce this by putting a newline in the start of the first variable:

$ a=$'\nfoo'
$ b='bar'
$ sed "s@$a@$b@"
sed: -e expression #1, char 2: unterminated `s' command

An unescaped newline terminates the sed command. Putting the newline later in a would of course give the error on a later character.

You did print out both variables earlier in the script, but they weren't quoted, so any leading whitespace in them would have been be removed, and whitespace in the middle would display as single spaces.

Check what your variables actually contain, with something like these:

printf ">%s<\n" "$to_replace"
printf "%q\n" "$to_replace"

The latter is a Bash feature that displays the string quoted in a way that Bash accepts as input. set -x would also show what goes to sed's command line, but you'd need to notice the literal newline from its output.

So, if a single leading newline is all there is, you could remove it in the beginning of the script:

to_replace=${pc#*,}
to_replace=${to_replace#$'\n'}

(You could combine those into one, but separate steps work even if the newline isn't there.)

Answer 2

Try this

sed -i -e "s@$to_replace@$replace_with@g" "$entry"

where

• -i means replace "in place", If you don't use -i replace will be on standard output.