0

We're having a very strange puzzle related to character encodings and bash replacement, where we are passing a bash variable into a rake script that gets piped into a different bash variable, and I'm not sure what to try next. It only works when you create the variable the non-replace way, but not only do the variables match either way you create them, but even their binary representations match. This should exclude anything relating to bad encodings, null-terminators or non-displayable characters. This is one of the most interesting bugs I've ever seen working with any language, so I wanted to share with the community and see if anyone has any ideas for what could be going on here. For further context:

RELEASE_CODE=1250
MATCHING=replace
NAME=redactedDeployment-replace-d

First, the working case in which the script passes with the expected result.

# FULLNAME=redactedDeployment-1250-d
# deployments=$(rake find_deployments[$FULLNAME])
# echo $deployments
redactedDeployment-1250-d
# echo $FULLNAME
redactedDeployment-1250-d
# echo $FULLNAME | perl -lpe '$_=join " ", unpack"(B8)*"'
01110010 01100101 01100100 01100001 01100011 01110100 01100101 01100100 01000100 01100101 01110000 01101100 01101111 01111001 01101101 01100101 01101110 01110100 00101101 00110001 00110010 00110101 00110000 00101101 01100100

and

# if [ "${MATCHING,,}" = "replace" ]; then
>       FULLNAME=${NAME/replace/$RELEASE_CODE}
>       tr -dc '[[:print:]]' <<< "$FULLNAME" #Remove non-printable characters, same result if we remove this line
>     else
>       FULLNAME=${NAME}-${RELEASE_CODE}
>     fi
# deployments=$(rake find_deployments[$FULLNAME])
rake aborted!
Don't know how to build task 'find_deployments[redactedDeployment-1250-d' (See the list of available tasks with `rake --tasks`)

(See full trace by running task with --trace)
# echo $FULLNAME
redactedDeployment-1250-d
# echo $FULLNAME | perl -lpe '$_=join " ", unpack"(B8)*"'
01110010 01100101 01100100 01100001 01100011 01110100 01100101 01100100 01000100 01100101 01110000 01101100 01101111 01111001 01101101 01100101 01101110 01110100 00101101 00110001 00110010 00110101 00110000 00101101 01100100
9
  • This line tr -dc '[[:print:]]' <<< "$FULLNAME" does not modify the FULLNAME variable in any way, so yeah, it's a no-op. Commented Jul 22, 2019 at 21:58
  • Check if your script file/rakefile contains \r\n line endings: cat -v file Commented Jul 22, 2019 at 21:59
  • @glennjackman Good to note, doesn't solve the puzzle at hand though with the exact binary matches (fed them thru diffchecker.com). I will add that this isn't urgent, I wound up using some sed magic to get it to take in a var not a file, but still very curious why the parameter expansion is broken in this strange way. Commented Jul 22, 2019 at 22:03
  • Are you 100% sure you're not just missing the trailing ] in your code for the argument you're passing to rake? Commented Jul 22, 2019 at 22:06
  • 1
    I think the "one of the most interesting bugs" is in fact user not quoting variables. It only needs $FULLNAME to have a trailing space or newline (probably originating in $RELEASE_CODE) to trigger this kind of behavior. As said, tr is a no-op. And (because of lack of quoting) echo $FULLNAME may "hide" the very same characters that are the culprit elsewhere (because of lack of quoting there). Commented Jul 22, 2019 at 23:46

0

You must log in to answer this question.

Browse other questions tagged .