Why does Parallel.For execute the WinForms message pump, and how to prevent it?

Question

I'm trying to speed up a lengthy (a few ms) operation* using Parallel.For, but I'm getting Paint events all over my WinForms application before the method has returned - suggesting it somehow triggers a message pump. The overall redraw, however, leads to accessing data in an inconsistent state, producing erratic errors and exceptions. I need to assure that Parallel.For, while blocking, doesn't trigger UI code.

My research on this so far has been inconclusive and pointed me roughly to things like synchronization contexts and TaskScheduler implementations, but I have yet to make sense of it all.

If someone could help me along the way by clearing some things up, that would very much be appreciated.

1. What is the chain of events that leads to Parallel.For triggering WinForms message pump?
2. Is there any way I can prevent this from happening entirely?
3. Alternatively, is there any way to tell if a UI event handler is called from the regular message pump, or the "busy" message pump as triggered by Parallel.For?

---

Edit: * Some context: The above few ms operation is part of a game engine loop, where 16 ms are available for a full update - hence the attribute "lengthy". The context of this problem is executing a game engine core inside its editor, which is a WinForms application. Parallel.For happens during the internal engine update.

Answer 1

This comes from the CLR, it implements the contract that an STA thread (aka UI thread) is never allowed to block on a synchronization object. Like Parallel.For() does. It pumps to ensure that no deadlock can occur.

This gets Paint events to fire, and some others, the exact message filtering is a well-kept secret. It is pretty similar to DoEvents() but the stuff that is likely to cause re-entrancy bugs blocked. Like user input.

But clearly you have a DoEvents() style bug in spades, re-entrancy is forever a nasty bug generator. I suspect you'll need to just set a bool flag to ensure that the Paint event skips an update, simplest workaround. Changing the [STAThread] attribute on the Main() method in Program.cs to [MTAThread] is also a simple fix, but is quite risky if you also have normal UI. Favor the private bool ReadyToPaint; approach, it is simplest to reason through.

You should however investigate exactly why Winforms thinks that Paint is needed, it shouldn't since you are in control over the Invalidate() call in a game loop. It may fire because of user interactions, like min/max/restoring the window but that should be rare. Non-zero odds that there's another bug hidden under the floor mat.

Answer 2

As already explained, Parallel.For itself does not execute the WinForms message pump, but the CLR implementation of Wait which is called by the necessary thread synchronization primitives is causing the behavior.

Luckily that implementation can be overridden by installing a custom SynhronizationContext because all CLR waits actually call Wait method of the current (i.e. associated with the current thread) synchronization context.

The idea is to call WaitForMultipleObjectsEx API which has no such side effects. I can't say whether it is safe or not, CLR designers have their reasons, but from the other side, they have to handle many different scenarios which may not apply to your case, so at least it's worth trying.

Here is the class:

using System;
using System.ComponentModel;
using System.Runtime.InteropServices;
using System.Security;
using System.Threading;
using System.Windows.Forms;

class CustomSynchronizationContext : SynchronizationContext
{
    public static void Install()
    {
        var currentContext = Current;
        if (currentContext is CustomSynchronizationContext) return;
        WindowsFormsSynchronizationContext.AutoInstall = false;
        SetSynchronizationContext(new CustomSynchronizationContext(currentContext));
    }

    public static void Uninstall()
    {
        var currentContext = Current as CustomSynchronizationContext;
        if (currentContext == null) return;
        SetSynchronizationContext(currentContext.baseContext);
    }

    private WindowsFormsSynchronizationContext baseContext;

    private CustomSynchronizationContext(SynchronizationContext currentContext)
    {
        baseContext = currentContext as WindowsFormsSynchronizationContext  ?? new WindowsFormsSynchronizationContext();
        SetWaitNotificationRequired();
    }

    public override SynchronizationContext CreateCopy() { return this; }
    public override void Post(SendOrPostCallback d, object state) { baseContext.Post(d, state); }
    public override void Send(SendOrPostCallback d, object state) { baseContext.Send(d, state); }
    public override void OperationStarted() { baseContext.OperationStarted(); }
    public override void OperationCompleted() { baseContext.OperationCompleted(); }

    public override int Wait(IntPtr[] waitHandles, bool waitAll, int millisecondsTimeout)
    {
        int result = WaitForMultipleObjectsEx(waitHandles.Length, waitHandles, waitAll, millisecondsTimeout, false);
        if (result == -1) throw new Win32Exception();
        return result;
    }

    [SuppressUnmanagedCodeSecurity]
    [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
    private static extern int WaitForMultipleObjectsEx(int nCount, IntPtr[] pHandles, bool bWaitAll, int dwMilliseconds, bool bAlertable);
}

In order to activate it, just add the following line before your Application.Run(...) call:

CustomSynchronizationContext.Install();

Answer 3

Hans explained it in. So what should you do? The easiest path would be to have one volatile bool flag that says if data are consistent and is okay to use them to paint.

Better, but more complicated, solution would be to replace Parallel.For with your own ThreadPool, and send simple task to the pool. Main GUI thread would then stay responsive to user input.

Also, those simple task must not change the GUI directly, but just manipulate the data. Game GUI must be changed only in OnPaint.

Answer 4

Hans explained it. So what should you do? Don't run that loop on the UI thread. Run it on a background thread, for example:

await Task.Run(() => Parallel.For(...));

Blocking on the UI thread is not a good idea in general. Not sure how relevant that is to a game engine loop design but this fixes the reentrancy problems.