How does the instruction decoder differentiate between EVEX prefix and BOUND opcode in 32-bit mode?

Question

In 32-bit mode Intel solves the VEX prefix vs LDS/LES conflict by inverting the high bits of register extension, because the mod field of ModRM byte can\'t be 11b

Answer 1

The trick with inverted bits works because of two facts:

1. In the 32-bit mode, only 8 of 32 ZMM registers are available, therefore EVEX.R' bit never flips, and thus never aliases with allowed BOUND encodings.
2. In the 64-bit mode, there is no BOUND instruction recognized so the EVEX bits can take any value.

An excerpt from a patent on AVX clarifies this:

[0111] REX′ field —this is the first part of the REX′ field and is the EVEX.R′ bit field (EVEX Byte 1, bit [4]—R′) that is used to encode either the upper 16 or lower 16 of the extended 32 register set. In one embodiment of the invention, this bit, along with others as indicated below, is stored in bit inverted format to distinguish (in the well-known x86 32-bit mode) from the BOUND instruction, whose real opcode byte is 62, but does not accept in the MOD R/M field (described below) the value of 11 in the MOD field; alternative embodiments of the invention do not store this and the other indicated bits below in the inverted format. A value of 1 is used to encode the lower 16 registers. In other words, R′Rrrr is formed by combining EVEX.R′, EVEX.R, and the other RRR from other fields.

However, the Intel SDM is unclear on this fact. I've looked through the SDM and indeed, in the EVEX section there are no traces of mentioning of the complement meaning of EVEX encodings. One must deduce it somehow from the fact that EVEX is an extension of VEX, and for the latter there is a statement of inverted meaning (volume 2A, section 2.3.5, the first bullet):

This field is encoded using 1’s complement form (inverted form), i.e. XMM0/YMM0/R0 is encoded as 1111B, XMM15/YMM15/R15 is encoded as 0000B.