The implementation of this Set
is far simpler than HashSet
as it is only needs to add and remove elements and check for existence for LINQ internal processes. It does not implement any interfaces or expose iterators etc.
So probably it is faster for the purpose LINQ uses it for.