Win32: How to make drop shadow honor non-rectangular Layered window?

Question

i've created a layered window by adding the the WS_EX_LAYERED extended style:

wndClass.ExStyle = wndClass.ExStyle | WS_EX_LAYERED;

Windows will use black as the chroma key color value. i'm going to leave a large border of black to make the problem obvious:

---

After the window is constructed, i tell it to use black as a chroma-key color:

SetLayeredWindowAttributes(hwnd, 0x00000000, 255, LWA_COLORKEY);

Now the popup layered window appears partially transparent:

---

The problem is the final step. i want to use CS_DROPSHADOW class style, available since Windows XP, to create a drop-shadow:

wndClass.Style = wndClass.Style | CS_DROPSHADOW;

The drop shadow appears, but the shadow surrounds the original rectangular window, and doesn't take into account the window's transparency provided by the layered window:

Does anyone know what magical option i've missed somewhere that will make the drop shadow honor the non-rectangular layered window?

---

Another example of where this issue appears is when you don't include the 6px padding/margin. The hint window as drawn by Windows® themes is non-rectangular. This leaves a small visible gap where the window is transparent, but the drop shadow does not appear:

Microsoft has managed to make it work, as you can see from this hint from Internet Explorer:

---

Looking closer at a Windows tooltips class hint window. Using SpyXX - i can get its window rect, and class styles:

SpyXX says:

Rectangle:     (440, 229)-(544, 249), 104x20
Restored Rect: (440, 229)-(544, 249), 104x20
Client Rect:   (0, 0)-(104, 20), 104x20

So everything points to the window itself being 104x20 pixels, with the drop shadow outside the window itself. (Which is consistent with CS_DROPSHADOW.)

Next i can look at the styles of the tooltips window class:

Windows Styles:     94000001

    WS_POPUP        80000000
    WS_VISIBLE      10000000
    WS_CLIPSIBLINGS  4000000
    TTS_ALWAYSTIP          1

Extended Styles:    00080088

    WS_EX_LAYERED      80000
    WS_EX_TOOLWIN         80
    WS_EX_TOPMOST          8

Interestingly, it doesn't use CS_SAVEBITS (0x800); which is useful for small, short-lived, windows.

Nor does it use CS_DROPSHADOW (0x20000). So now i wonder how is it drawing outside its own window?

Note: Transparent layered windows is documented as the preferred technique over regions.

Edit: Layered Windows have been around with Windows 2000. CS_DropShadow was added with XP.

Answer 1

Transparent layered windows is documented as the preferred technique over regions.

However, CS_DROPSHADOW does pay attention to regions. If you crop or otherwise shape your window using a region, the drop-shadow will follow the new outline.

Fortunately, you can use regions with layered windows, and by combining the two get the effect you're looking for.

BTW: tooltips_class32 does use CS_DROPSHADOW - you won't see it in the window styles because it's a class style, not a window style.

Answer 2

Why don't you use LWA_ALPHA and build the shadow into the image?

Edit in reponse to your comment:

A) Doesn't stop you using an alpha channeled PNG for a shadow only. Blt the 2 images together and use as one single image.
B) Its not hard to generate a drop shadow. In the image you posted its black with 3 different alpha values.
C) But it doesn't work does it? ie Time to get creative.
D) Then don't try and get windows to do something it won't do for you.
E) Is entirely irrelevant. Layered windows handle that for you.

I strongly recommend you learn more about layered windows because they CAN help you to your goal.

Edit2: You have the bitmap. Its fairly easy to scan over the image and find which bits will be colour keyed (by identifying the black yourself) and hen modify that to have an alpha of 0 where everything else will have an alpha of 255 (Not: You may have to convert the image to a 32-bit image from a lower colour format, you can do this by creating a new DC and copying the image). This will give you the same effect with LWA_ALPHA as with LWA_COLORKEY. From there its fairly easy to identify the pixel at the edge, where the color changes to (R = 0, G = 0, B = 0, A = 0). You then change that first pixel to have a n alpha of 192, the one blow it to 128 and the one below to 64. You now have an alpha'd gradation below the image that will look like the shadow. You can adjust the alpha to get the effect you want.

Answer 3

CS_DROPSHADOW only works with standard rectangular windows. WS_EX_LAYERED windows don't support everything. They are more of a low-level, self-service method to draw exactly what you want.

To get a drop shadow, you'll have to generate the drop-shadow yourself from the alpha channel in the image.