You will only get InvalidOperationException: Collection was modified
if you enumerate the dictionary while modifying.
However, that is not thread-safe.
If one of those operations causes the dictionary to resize, the other one may get lost.
Instead, use ConcurrentDictionary
.