Custom dwm drawn window frame flickers on resizing if the window contains a HwndHost element

Question

I've been thinking about this for a couple of days but I think I lack some basic understanding of how windows and wpf work internally to figure this out.

The problem is this:

I created a window that should let me draw wpf controls on an aero title bar (like office). This works fine as long as I don't add a Hwndhost element to the window, in this case whenever i resize it the frame and the HwndHost start to flicker pretty badly (other elements seem to render properly). I also tried using the custom frame window implementation from the WPF Shell Integration library and the result is the same, so I'm thinking that it's not entirely my fault.

The following code is a simple compilable program that reproduces the problem. The sample is in c# but the answer doesn't have to be.

using System;
using System.Runtime.InteropServices;
using System.Windows;
using System.Windows.Controls;
using System.Windows.Interop;
using System.Windows.Media;
using System.Windows.Threading;

namespace DwmTest {
    class Program {
        [STAThread]
        static void Main( ) {
            var w = new CustomFrameWindow{ Content =  new WindowHost() };
            w.Show( );
            ((Border)VisualTreeHelper.GetChild( w, 0 )).Margin = new Thickness( 11, 33, 11, 11 );
            Dispatcher.Run( );
        }
    }

    public class CustomFrameWindow : Window {

        const int resizeFrameWidth = 11;
        const int captionHeight = 33;

        public enum HT { CLIENT = 1, CAPTION = 2, LEFT = 10, RIGHT, TOP, TOPLEFT, TOPRIGHT, BOTTOM, BOTTOMLEFT, BOTTOMRIGHT }

        [StructLayout( LayoutKind.Sequential )]
        public struct Margins { public int left, right, top, bottom; }

        [DllImport( "user32.dll" )]
        public static extern bool SetWindowPos( IntPtr hWnd, IntPtr hWndInsertAfter, int x, int y, int cx, int cy, int flags );

        [DllImport( "dwmapi.dll" )]
        public static extern bool DwmDefWindowProc( IntPtr hwnd, int msg, IntPtr wParam, IntPtr lParam, out IntPtr result );

        [DllImport( "dwmapi.dll", PreserveSig = false )]
        public static extern void DwmExtendFrameIntoClientArea( IntPtr hwnd, ref Margins pMarInset );

        protected override void OnSourceInitialized( EventArgs e ) {
            base.OnSourceInitialized( e );

            var hWndSource = HwndSource.FromHwnd( new WindowInteropHelper( this ).Handle );
            hWndSource.CompositionTarget.BackgroundColor = Colors.Transparent;

            var nonClientArea = new Margins{ 
                left = resizeFrameWidth, top = captionHeight, bottom = resizeFrameWidth, right = resizeFrameWidth
            };
            DwmExtendFrameIntoClientArea( hWndSource.Handle, ref nonClientArea );

            hWndSource.AddHook( WndProc );

            // FRAMECHANGED | NOMOVE | NOSIZE
            SetWindowPos( hWndSource.Handle, new IntPtr( ), 0, 0, 0, 0, 0x0020 | 0x0002 | 0x0001 );
        }

        private IntPtr WndProc( IntPtr hwnd, int msg, IntPtr wParam, IntPtr lParam, ref bool handled ) {

            switch( msg ) {
                case 0x0083: // NCCALCSIZE
                    if( wParam != IntPtr.Zero ) handled = true;
                    break;
                case 0x0084: // NCHITTEST
                    handled = true;

                    IntPtr dwmHitTest;
                    if( DwmDefWindowProc( hwnd, msg, wParam, lParam, out dwmHitTest ) ) {
                        return dwmHitTest;
                    }

                    var mousePosition = PointFromScreen( new Point( lParam.ToInt32( ) & 0xFFFF, lParam.ToInt32( ) >> 16 ) );

                    var isTop = mousePosition.Y <= resizeFrameWidth;
                    var isBottom = mousePosition.Y >= ActualHeight - resizeFrameWidth;
                    var isLeft = mousePosition.X <= resizeFrameWidth;
                    var isRight = mousePosition.X >= ActualWidth - resizeFrameWidth;

                    var hitTest = HT.CLIENT;
                    if( isTop ) {
                        if( isLeft ) hitTest = HT.TOPLEFT;
                        else if( isRight ) hitTest = HT.TOPRIGHT;
                        else hitTest = HT.TOP;
                    }
                    else if( isBottom ) {
                        if( isLeft ) hitTest = HT.BOTTOMLEFT;
                        else if( isRight ) hitTest = HT.BOTTOMRIGHT;
                        else hitTest = HT.BOTTOM;
                    }
                    else if( isLeft ) hitTest = HT.LEFT; 
                    else if( isRight ) hitTest = HT.RIGHT; 
                    else if( mousePosition.Y <= captionHeight ) hitTest = HT.CAPTION;

                    return new IntPtr( (int)hitTest );
            }
            return IntPtr.Zero;
        }
    }

    public class WindowHost : HwndHost {
        [DllImport( "user32.dll", SetLastError = true )]
        static extern IntPtr CreateWindowEx( IntPtr exStyle, string lpClassName,string lpWindowName,int dwStyle,int x,int y,int nWidth,int nHeight,IntPtr hWndParent,IntPtr hMenu,IntPtr hInstance,IntPtr lpParam );

        protected override HandleRef BuildWindowCore( HandleRef hWndParent ) {
            return new HandleRef( this, CreateWindowEx( IntPtr.Zero, "static", "", 0x40000000, 0, 0, 200, 200, hWndParent.Handle, IntPtr.Zero, IntPtr.Zero, IntPtr.Zero ) );
        }
        protected override void DestroyWindowCore( HandleRef hwnd ) { }
    }
}

Answer 1

I solved it by adding WS_CLIPCHILDREN as a style when CreatWindowEx

protected override HandleRef BuildWindowCore(HandleRef hwndParent)
{
    _hwndHost = Win32Api.CreateWindowEx(0, "Static", "", 
                        (int) (WS_CHILD | WS_VISIBLE | WS_CLIPCHILDREN),
                      0, 0,
                      _hostWidth, _hostHeight,
                      hwndParent.Handle,
                      IntPtr.Zero,
                      IntPtr.Zero,
                      0);

 }

Answer 2

Well, I finally found a fix but I think this one borders on black magic... anyway it turns out that responding to WM_NCCALCSIZE with any rect smaller than the window solves the problem. So for example changing the handler to look like below removes the flicker!

            case WM.NCCALCSIZE:
                if( wParam != IntPtr.Zero ) {
                    handled = true;
                    var client = (RECT)Marshal.PtrToStructure( lParam, typeof( RECT ) );
                    client.Bottom -= 1;
                    Marshal.StructureToPtr( client, lParam, false );
                }
                break;

I have no idea why this is working and I'm sure that a saner solution exists, so I'd be happy if someone could enlighten me.