I have a transparent BufferedImage created with the following code(not relevant how it is created, I think):
GraphicsEnvironment ge = GraphicsEnv
Got it :) used clearRect instead of fill with a transparent color.
graphics = (Graphics2D) offlineBuffer.getGraphics();
graphics.setBackground(new Color(255, 255, 255, 0));
Rectangle screen = transformationContext.getScreen();
graphics.clearRect(0,0, (int)screen.getWidth(), (int)screen.getHeight());
One relatively fast way, but I don't know if it's the fastest (and I'd like to see other answers) is to have another picture that you never modify and that is always "fully cleared" / "fully transparent" and then you do a raster copy, say you named that copy CLEAR:
imageYouWantToClear.setData( CLEAR.getRaster() );
Note that working with graphics can be very tricky when it comes to performances because there are a lot of not-very-well-documented behavior. For example your images (say the CLEAR one) may be hardware-accelerated but you'd then lose hardware-acceleration as soon as you'd use a mutating method (like say a setRgb()) and it would prove very difficult to realize that you just lost the benefit of hardware acceleration.
I think that the best place to find infos on the subject of performant BufferedImage would be in the Java game-programmers and Java game-API-programmers community/forums.
Btw make sure that both your BufferedImage are using the 'compatible' mode: TYPE_INT_ARGB may be fine on Windows but not on OS X, etc. so you want to create them doing something like:
GraphicsEnvironment.getLocalGraphicsEnvironment().getDefaultScreenDevice().getDefaultConfiguration().createCompatibleImage(width, height, Transparency.TRANSLUCENT);
Ouch the Law-of-Demeter hurts, thanks Java ;)