Because more conditions to check is more work for the database engine. Seems logical to me.
If you were to have one condition over a clustered index field, this particular check wouldn't slow down the operation that much. Have you considered rearranging indexes to match your query?