See exact content of bash variable? (hexdump doesn't help)

Question

So I have a problem below, but my question is more generic - how to see exact content of memory referenced by bash variable to understand why they don't match:

# [[ $c1 == $c ]] || echo nope
nope
# [[ $c1 == "bkup dt" ]] || echo nope
# [[ $c == "bkup dt" ]] || echo nope
nope
# hexdump -C <<<$c
00000000  62 6b 75 70 20 64 74 0a                           |bkup dt.|
00000008
# hexdump -C <<<$c1
00000000  62 6b 75 70 20 64 74 0a                           |bkup dt.|
00000008
# [ "$c1" = "$c" ] || echo nope
nope
# [ ! "$c1" = "$c" ] || echo nope

Or does it look like a bug? I can repeat the problem with:

$ cd /tmp
$ mkdir aaa
$ echo 2 > aaa/1
$ echo 2 > aaa/2
$ c=$(ls -A aaa)
$ [[ $c == $(echo $c) ]] || echo not equal
not equal
$ hexdump -C <<<$c
00000000  31 20 32 0a                                       |1 2.|
00000004
$ hexdump -C <<<$(echo $c)
00000000  31 20 32 0a                                       |1 2.|
00000004
$ c1="1 2"
$ [[ $c1 == $(echo $c1) ]] || echo not equal
$ [[ $c1 == $(echo $c) ]] || echo not equal
$ [[ $c1 == $c ]] || echo not equal
not equal

Answer 1

The best thing to inspect the content of a variable is to use declare -p:

$ c="1 2"
$ declare -p c
declare -- c="1 2"

Note that your tests are wrong because you're missing out quotes in your variable expansions!

Look:

$ c="1  2" # with two spaces
$ declare -p c
declare -- c="1  2"
$ echo $c
1 2
$ echo "$c"
1  2
$ d=$(echo $c)
$ declare -p d
1 2

You must quote every single variable expansion, unless you really want to have word splitting and pathname expansion applied to them! (and usually, you certainly don't want that to happen).

Even with your hexdump strategy you need quotes:

$ c="1  2" # two spaces
$ hexdump <<< $c
00000000  31 20 32 0a                                       |1 2.|
00000004
$ hexdump <<< "$c"
00000000  31 20 20 32 0a                                    |1  2.|
00000005

What you're experiencing is exactly this:

$ mkdir aaa; touch aaa/{1,2}
$ c=$(ls -A aaa)
$ declare -p c
declare -- c="1
2"
$ # see? there's a new line between the files.
$ echo $c
1 2
$ echo "$c"
1
2
$ # Use quotes!

---

Sometimes declare -p will not quite show spaces properly. In this case you can use printf like so:

$ c=$'    \n'
$ declare -p c
declare -- c="    
"
$ # there are spaces, but you can't see them
$ printf '%q\n' "$c"
$'    \n'

---

The declare strategy is also nice as you can inspect arrays and functions too:

$ a=( one two "three four" )
$ declare -p a
declare -a a='([0]="one" [1]="two" [2]="three four")'
$ declare -A h=( [one]=1 [two]=2 )
$ declare -p h
declare -A h='([one]="1" [two]="2" )'
$ f() { echo hello; } > somewhere > >(over the rainbow)
$ declare -pf f
f () 
{ 
    echo hello
} > somewhere 2> >(over the rainbow)
$ # You need also the -f switch to target functions

You also have access to the flags of the variables:

$ declare -litux hello=world
$ declare -p hello
declare -itx hello="0"
$ # Have fun!

Answer 2

You don't see the exact content of the variable because you ask bash to modify it. The most basic effect is in this command:

$ echo hello # world
hello

$ echo "hello # world"
hello # world

Quoting prevents the shell from interpreting the special character # as a comment start character, and thus it gets printed.

Please read more here

The only correct way to print a variable exact content is to quote it.

a='a \b 
|c     d'   

Above, var a is getting an exact string because it is single quoted.

$ echo "$a"
a \b 
|c     d

Above, the value of var a is exactly reproduced because of the double quotes.

As soon as you fail to quote a variable expansion $a, you get something else:

$ echo $a
a \b |c d

This is a very basic requirement of any shell: quoting.
And there are single quotes, double quotes and backslash quoting.

---

In your case, to reproduce what you see, the vars are not really equal, they are:

$ c='bkup dt
'

$ c1='bkup dt'

That is, c contains an additional newline at the end compared to c1.

Let's repeat your commands, but correctly quoting to see the real contents of each var (Please understand that inside [[…]] variables could be used un-quoted most of the time).

$ [[ $c1 == $c ]] || echo nope
nope

The variables are Not equal.

$ [[ $c1 == "bkup dt" ]] || echo nope

variable c1 is exactly equal to bkup dt (no output).

$ [[ $c == "bkup dt" ]] || echo nope
nope

No, c is not exactly "bkup dt" (it has an additional new line).

$ hexdump -C <<<"$c"
00000000  62 6b 75 70 20 64 74 0a  0a                       |bkup dt..|
00000009

Note the two newlines at the end, now the quoted variable reproduces its internal value correctly with an added newline from the <<<.

Compare with $c1.

$ hexdump -C <<<"$c1"
00000000  62 6b 75 70 20 64 74 0a                           |bkup dt.|
00000009

Only one newline!!.

Maybe a better way will be to use:

$ printf '%s' "$c" | od -vAn -tx1c
 62  6b  75  70  20  64  74  0a
  b   k   u   p       d   t  \n

Here, the newline inside the variable is clearly shown.

The variable $c1 doesn't show that:

$ printf '%s' "$c1" | od -vAn -tx1c
 62  6b  75  70  20  64  74
  b   k   u   p       d   t

The command declare could be used also:

$ declare -p c
declare -- c="bkup dt
"

$ declare -p c1
declare -- c1="bkup dt"

As cat, sed and some others:

$ echo "$c" | cat -vET
bkup dt$
$

$ echo "$c1" | cat -vET
bkup dt$

$  echo "$c" | sed -n l
bkup dt$
$

$  echo "$c1" | sed -n l
bkup dt$

Provided you quote the variable expansion and know how a new line looks like.

---

Your second example also has a new line issue.

$ cd /tmp; mkdir aaa; echo 2 > aaa/1; echo 2 > aaa/2
$ c=$(ls -A aaa)
$ echo $c | od -vAn -tx1c
  31  20  32  0a
   1       2  \n

In this case, c seem to have an space, but what it really have is:

$ echo "$c" | od -vAn -tx1c
  31  0a  32  0a
   1  \n   2  \n

A newline

Thus, an echo also fails (for a couple of missing quotes, both are needed):

$ echo $(echo $c) | odc
  31  20  32  0a
   1       2  \n

$ echo "$(echo "$c")" | odc
  31  0a  32  0a
   1  \n   2  \n