Conflicting signs in x86 assembly: movsx then unsigned compare/branch?

Question

I am confused in the following snippet:

movsx   ecx, [ebp+var_8] ; signed move
cmp     ecx, [ebp+arg_0]
jnb     short loc_401027 ; unsigned jump

Answer 1

As noted by Jester, unsigned comparison can be used to do range checks for signed numbers. For example, a common C expression that checks whether an index is between 0 and some limit:

short idx = ...;
int limit = ...; // actually, it's called "arg_0" - is it a function's argument?
if (idx >= 0 && idx < limit)
{
    // do stuff
}

Here idx, after sign-extension, is a signed 32-bit number (int). The idea is, when comparing it with limit as if it were unsigned, it does both comparisons at once.

1. If idx is positive, then "signed" or "unsigned" doesn't matter, so unsigned comparison gives the correct answer.
2. If idx is negative, then interpreting it as an unsigned number will yield a very big number (greater than 2³¹-1), so in this case, unsigned comparison also gives the correct answer.

So one unsigned comparison does the work of two signed comparisons. This only works when limit is signed and non-negative. If the compiler can prove it's non-negative, it will generate such optimized code.

---

Another possibility is if the initial C code is buggy and it compares signed with unsigned. A somewhat surprising feature of C is that when a signed variable is compared with unsigned, the effect is unsigned comparison.

short x = ...;
unsigned y = ...;

// Buggy code!
if (x < y) // has surprising behavior for e.g. x = -1
{
    // do stuff
}

if (x < (int)y) // better; still buggy if the casting could overflow
{
    // do stuff
}