How far IEEE 754 can be violated for the goal of performance increase?

Question

As we know, the practice "not fully IEEE 754 conformant, but faster" exists. This is what compilers optionally do (options related to floating-point model