VBA: Why would the Not operator stop working? [duplicate]

Answer 1

It's because of the internal Long value coming from this property, as explained by Cindy Meister. We should always use CInt to avoid this.

Sub testChangeBoolean2()
  Dim X As Boolean                     ' again, default is False
  X = CInt(Selection.Cells(1).FitText) ' [Fixed] does set X to True
  X = Not X                            ' X is False!
End Sub

Answer 2

I believe the explanation has to do with how older programming languages (WordBasic and early VBA) stored the integer values of True and False. In those days, True = -1 and False = 0.

Newer programming languages still use 0 for False, but 1 for True.

The majority of Word's Boolean type properties continue to use -1 for True (Font.Bold, for example), which has been cause for confusion and frustration for programmers working with the Interop in newer languages. So, at some point, some developers at Microsoft decided to use the new way and assigned the integer value of 1 to True for some new functionality. Such as FitText.

Considering the following code sample, where X is of type Boolean and y of type Integer:

• If FitText is True, the integer value is 1
• If reversing the values, using Not shows that the Boolean remains "True" because its integer value is not 0, it's -2
• Setting the integer value directly to True gives -1

This is confusing, indeed, but does explain why Not is not giving the expected result.

Sub testChangeBoolean()
  Dim X As Boolean                  ' again, default is False
  Dim Y As Integer
  X = Selection.Cells(1).FitText    ' does set X to True
  Y = Selection.Cells(1).FitText
  Debug.Print X, Y                  ' result: True    1
  X = Not X                         ' X is still True!
  Y = Not Y
  Debug.Print X, Y                  ' result: True   -2
  X = False
  Y = True
  Debug.Print X, Y                  ' result: False  -1
End Sub

Answer 3

First of all, the Not operator works properly, exactly as it should.

"Duh. But I'm trying to toggle the .FitText property in a table cell (in Word). .FitText starts as True..."

To toggle, this works just fine:

If Selection.Cells(1).FitText Then
    Selection.Cells(1).FitText = False
Else
    Selection.Cells(1).FitText = True
End If

As for why you do not see the result you expected, this has to do with type casting, implicit type conversions and passing value of a type to a variable of the same type. In your case, the cell's property is Boolean. Since you explicitly set the receiving variable type to Boolean, no implicit type conversion is taking place: Boolean X gets the internal value of the Boolean property, which (Cindy Meister is correct) just happened to be 1, and 1 does evaluate to True as a Boolean variable.

Now, Mathieu Guindon correctly commented that "The -2 is because logical operators do bitwise operations...". So, Not 1 = -2, and -2 also evaluates to True; in fact, any number stored internally for a Boolean variable but 0 makes the value of this variable True, as in an explicit CBool(any_number_but_0) = True.

So the problem here is Boolean-to-Boolean passing of the numeric value 1 (the internal representation) from one to another. You need to force variable type re-evaluation, implicitly or explicitly, and all will work as you wanted:

' Illustration: a few possible solutions
Dim X As Variant  
' implicit re-evauation
X = Selection.Cells(1).FitText ' X=True, a Variant sub-type Boolean,  CInt(X)=-1
X = Not X                      ' X=False, CInt(X)=0 

Dim X As Boolean  
' explicit re-evaluation
X = Selection.Cells(1).FitTex  ' X=True, CInt(X)=1
X = Not CBool(CInt(X))         ' X=False, CInt(X)=0

' explicit re-evaluation    
X = Selection.Cells(1).FitText ' X=True, CInt(X)=1
X = Not CBool(CStr(X))         ' X=False, CInt(X)=0

' explicit and implicit (by the "+" operator) re-evaluation
X = Selection.Cells(1).FitText ' X=True, CInt(X)=1
X = Not (CBool(X) + 0)         ' X=False, CInt(X)=0; same as Not (True + 0) 

etc.

Answer 4

To add on to Cindy's excellent answer, I want to point out that while VBA normally has safeguards to coerce the values when assigning to a Boolean data type, this can be circumvented. Basically, if you write a random value to a memory address that's not yours, then you should expected undefined behavior.

To help demonstrate this, we'll (ab)use LSet which essentially allow us to copy the value without actually assigning.

Private Type t1
  b As Boolean
End Type

Private Type t2
  i As Integer
End Type

Private Sub Demo()
  Dim i1 As t2
  Dim b1 As t1
  Dim b As Boolean

  i1.i = 1

  LSet b1 = i1

  b = b1.b

  Debug.Print b, b1.b, i1.i
  Debug.Print CInt(b), CInt(b1.b), i1.i

End Sub

Note the line b = b1.b is basically equivalent to what we did in the OP code

X = Selection.Cells(1).FitText

That is, assigning a Boolean to another Boolean. However, because I wrote to the b1.b using LSet, bypassing VBA runtime checks, it doesn't get coerced. When reading the Boolean, VBA does implicitly coerce it into either True or False, which seems misleading but is correct because any falsy results is one that equals 0 (aka False), and any truthy results is one that doesn't. Note that the negative for truthy means that both 1 and -1 are truthy.

Had I assigned the 1 to a Boolean variable directly, VBA would have had coerced it into -1/True and thus there'd be no problem. But evidently with FitText or LSet, we are basically writing to the memory address in an uncontrolled fashion, so that VBA start to behave strangely with this particular variable since it expects the Boolean variable to already had its contents coerced but wasn't.