I used a WebView and loaded the image from the memory via
webview.loadUrl("file://...")
The WebView handles all the panning zooming and scrolling. If you use wrap_content the webview won't be bigger then the image and no white areas are shown.
The WebView is the better ImageView ;)