Re: Non-rectangular windows

[Nicolas Roard]

On Fri, Aug 8, 2008 at 4:05 PM, Christian Stieber <cstieber@webtogo.de> wrote:
> Hi,
>
> well, I have an application that uses a lot of non-rectangular windows
> on the Mac, created by using an NSView as content view that drawRect:-s
> an NSImage:
>
> -(void)drawRect:(NSRect)Rectangle
>
> {
>  (void)Rectangle;
>
>  if (SkinImage!=nil)
>    {
>      [[NSColor clearColor] set];
>      NSRectFill([self frame]);
>
>      [SkinImage compositeToPoint:NSZeroPoint
> operation:NSCompositeSourceOver];
>
> #ifdef APPLE
>      [[self window] invalidateShadow];
> #endif
> }
>
> Now, that didn't work with GNUstep. Originally I didn't think much about
> that
> since a lot of things don't work with GNUstep and need to be worked around
> in the application, so I started researching the possibilities to create
> non-
> rectangular windows on X11, stumbled across the Shape extension, and tried
> to
> use that --- which didn't work for some reason. Then I found that GNUstep
> actually DOES use the Shape extension...
>
> Since then, I've been trying to figure out how to get it to work. I did find
> that I need to
>  [Defaults setBool:YES forKey:@"XWindowBuffer-shape-hack"]
> in my GNUstep-init hacking, so it even considers using Shape, but it still
> doesn't seem to do anything.
>
> I did, however, succeed in setting a breakpoint on some function in the
> XWindowBuffer.m, set "has_alpha" to 1 in the debugger --- and I get a shaped
> window. Since then, I've been trying to figure out how to get GNUstep to set
> has_alpha... with no success.
>
> I tried randomly setting all sorts of "setBackgroundColor" and stuff to
> NSColor clearColor, looking for anything that looks like it might affect
> alpha handling, but no success. On the other hand, I don't really know
> where to look for what...
>
> Concerning GNUstep, only method that seems to set has_alpha is needsAlpha,
> and it only gets called from some ART file... but there it ends for me, I
> haven't been able to determine what any of the stuff there does or when it
> does what, so trying to backtrack it to the "real" NS* classes ended long
> before I got anywhere near that.
>
> I'm using gnustep-startup-0.19.2.tar.gz, if that helps. My build system
> knows how to unpack, modify and build from a startup archive, and last time
> I checked that was still the most current one. Since the above experiment
> with the debugger was with that version, shape support seems to be there;
> I just need to know the magic switch (if all else fails I'll just try to
> patch the check for has_alpha out of the library and see what happens then).
>
> Hints, anyone? :-)

IIRC, you are doing the right things: use NSColor clearColor to delete
the background, and sets the defaults for the shape extension. I know
it used to work with the art backend, so it may simply be the case
that the backend you are using (cairo?) does not support this method.
Check if your code works with the art backend, if it does then file a
bug :)

-- 
Nicolas Roard
"I love deadlines. I like the whooshing sound
they make as they fly by." -- Douglas Adams