MFMailComposeViewController image orientation

Question

I\'m developing a Universal app and I\'m coding for iOS6.

I\'m using the imagePickerController to take a photo and then I am sending it as an attachment using MFMail

Answer 1

swift version of the scale part:

class func scaleAndRotateImageUsingOrientation(imageIn : UIImage) -> UIImage
    {
        //takes into account the stored rotation on the image fromt he camera and makes it upright
        let kMaxResolution = 3264; // Or whatever

        let imgRef    = imageIn.CGImage
        let width     = CGImageGetWidth(imgRef)
        let height    = CGImageGetHeight(imgRef)
        var transform = CGAffineTransformIdentity
        var bounds    = CGRectMake( 0.0, 0.0, CGFloat(width), CGFloat(height) )

        if ( width > kMaxResolution || height > kMaxResolution )
        {
            let ratio : CGFloat = CGFloat(width) / CGFloat(height);

            if (ratio > 1)
            {
                bounds.size.width  = CGFloat(kMaxResolution)
                bounds.size.height = bounds.size.width / ratio;
            }
            else
            {
                bounds.size.height = CGFloat(kMaxResolution)
                bounds.size.width  = bounds.size.height * ratio
            }
        }

        let scaleRatio : CGFloat = bounds.size.width / CGFloat(width)
        var imageSize    = CGSizeMake( CGFloat(CGImageGetWidth(imgRef)), CGFloat(CGImageGetHeight(imgRef)) )
        let orient       = imageIn.imageOrientation;
        var boundHeight : CGFloat = 0.0

        switch(orient)
        {
        case UIImageOrientation.Up:                                        //EXIF = 1
            transform = CGAffineTransformIdentity;
            break;

        case UIImageOrientation.UpMirrored:                                //EXIF = 2
            transform = CGAffineTransformMakeTranslation(imageSize.width, 0.0);
            transform = CGAffineTransformScale(transform, -1.0, 1.0);
        break;

        case UIImageOrientation.Down:                                      //EXIF = 3
            transform = CGAffineTransformMakeTranslation(imageSize.width, imageSize.height);
            transform = CGAffineTransformRotate(transform, CGFloat(M_PI));
            break;

        case UIImageOrientation.DownMirrored:                              //EXIF = 4
            transform = CGAffineTransformMakeTranslation(0.0, imageSize.height);
            transform = CGAffineTransformScale(transform, 1.0, -1.0);
            break;

        case UIImageOrientation.LeftMirrored:                              //EXIF = 5
            boundHeight = bounds.size.height;
            bounds.size.height = bounds.size.width;
            bounds.size.width = boundHeight;
            transform = CGAffineTransformMakeTranslation(imageSize.height, imageSize.width);
            transform = CGAffineTransformScale(transform, -1.0, 1.0);
            transform = CGAffineTransformRotate(transform, 3.0 * CGFloat(M_PI) / 2.0);
            break;

        case UIImageOrientation.Left:                                      //EXIF = 6
            boundHeight = bounds.size.height;
            bounds.size.height = bounds.size.width;
            bounds.size.width = boundHeight;
            transform = CGAffineTransformMakeTranslation(0.0, imageSize.width);
            transform = CGAffineTransformRotate(transform, 3.0 * CGFloat(M_PI) / 2.0);
            break;

        case UIImageOrientation.RightMirrored:                             //EXIF = 7
            boundHeight = bounds.size.height;
            bounds.size.height = bounds.size.width;
            bounds.size.width = boundHeight;
            transform = CGAffineTransformMakeScale(-1.0, 1.0);
            transform = CGAffineTransformRotate(transform, CGFloat(M_PI) / 2.0);
            break;

        case UIImageOrientation.Right:                                     //EXIF = 8
            boundHeight = bounds.size.height;
            bounds.size.height = bounds.size.width;
            bounds.size.width = boundHeight;
            transform = CGAffineTransformMakeTranslation(imageSize.height, 0.0);
            transform = CGAffineTransformRotate(transform, CGFloat(M_PI) / 2.0);
            break;

        default:
            break

        }

        UIGraphicsBeginImageContext( bounds.size );

        var context = UIGraphicsGetCurrentContext();

        if ( orient == UIImageOrientation.Right || orient == UIImageOrientation.Left )
        {
            CGContextScaleCTM(context, -scaleRatio, scaleRatio);
            CGContextTranslateCTM(context, CGFloat(-height), 0);
        }
        else
        {
            CGContextScaleCTM(context, scaleRatio, -scaleRatio);
            CGContextTranslateCTM(context, 0, CGFloat(-height));
        }

        CGContextConcatCTM( context, transform );

        CGContextDrawImage( UIGraphicsGetCurrentContext(), CGRectMake( 0, 0, CGFloat(width), CGFloat(height) ), imgRef );
        let imageCopy = UIGraphicsGetImageFromCurrentImageContext();
        UIGraphicsEndImageContext();

        return( imageCopy );
    }

Answer 2

I have struggled with image orientations in iOS SDK7 my self and I want to add something to this discussion in the hope it is useful to others. UIImageJPEGRepresentation works fine, but be aware that using UIImagePNGRepresentation might result in an image with incorrect orientation.

I'm developing an app in which a picture is uploaded after being taken with the device camera. I do not store the image temporally to disk, but rather send it directly to the server.

When rendering a UIImage to NSData, you can choose between UIImagePNGRepresentation and UIImageJPEGRepresentation.

When using UIImagePNGRepresentation, both a picture shot in portrait and landscape orientation result in a landscape image. In this case the portrait image is rotated 90 degrees counter clockwise.

When using UIImageJPEGRepresentation, both a portrait and landscape image result with a correct orientation.

Answer 3

You need to rotate the image according to its orientation. Here's a trick how to do it easily:

NSData *data = UIImagePNGRepresentation(sourceImage);
UIImage *tmp = [UIImage imageWithData:data];
UIImage *fixed = [UIImage imageWithCGImage:tmp.CGImage
                                     scale:sourceImage.scale
                               orientation:sourceImage.imageOrientation];

Answer 4

In case anyone ends up here and requires a Swift 4 solution:

//  UIImage+orientedUp.swift
//  ID Fusion Software Inc
//  Created by Dave Poirier on 2017-12-27
//  Unlicensed
//

import UIKit

extension UIImage {
    func orientedUp() -> UIImage {
        guard self.imageOrientation != UIImageOrientation.up,
            let cgImage = self.cgImage,
            let colorSpace = cgImage.colorSpace
            else { return self }

        guard let ctx: CGContext = CGContext(data: nil,
                                             width: Int(self.size.width), height: Int(self.size.height),
                                             bitsPerComponent: cgImage.bitsPerComponent, bytesPerRow: 0, space: colorSpace,
                                             bitmapInfo: CGImageAlphaInfo.premultipliedLast.rawValue) else { return self }
        ctx.concatenate(self.orientedUpTransform())

        switch self.imageOrientation {
        case UIImageOrientation.left, UIImageOrientation.leftMirrored, UIImageOrientation.right, UIImageOrientation.rightMirrored:
            ctx.draw(cgImage, in: CGRect(x: 0, y: 0, width: self.size.height, height: self.size.width))
        default:
            ctx.draw(cgImage, in: CGRect(x: 0, y: 0, width: self.size.width, height: self.size.height))
        }

        guard let orientedCGImage: CGImage = ctx.makeImage()
            else { return self }

        return UIImage(cgImage: orientedCGImage)
    }

    func orientedUpTransform() -> CGAffineTransform {
        var transform: CGAffineTransform = CGAffineTransform.identity

        switch self.imageOrientation {
        case UIImageOrientation.down, UIImageOrientation.downMirrored:
            transform = transform.translatedBy(x: self.size.width, y: self.size.height)
            transform = transform.rotated(by: CGFloat(Double.pi))
        case UIImageOrientation.left, UIImageOrientation.leftMirrored:
            transform = transform.translatedBy(x: self.size.width, y: 0)
            transform = transform.rotated(by: CGFloat(Double.pi / 2.0))
        case UIImageOrientation.right, UIImageOrientation.rightMirrored:
            transform = transform.translatedBy(x: 0, y: self.size.height)
            transform = transform.rotated(by: CGFloat(-Double.pi / 2.0))
        default:
            break
        }

        switch self.imageOrientation {
        case UIImageOrientation.upMirrored, UIImageOrientation.downMirrored:
            transform = transform.translatedBy(x: self.size.width, y: 0)
            transform = transform.scaledBy(x: -1, y: 1)
        case UIImageOrientation.leftMirrored, UIImageOrientation.rightMirrored:
            transform = transform.translatedBy(x: self.size.height, y: 0)
            transform = transform.scaledBy(x: -1, y: 1)
        default:
            break
        }

        return transform
    }
}

You can then use it like this:

let image = UIImage(data: imageData)?.orientedUp()

Answer 5

I've spent a number of hours working on this issue and I am now clear about what's going on and how to fix it.

To repeat, the problem I ran into is:

When I shoot an image in portrait mode on the camera using the imagePickerController and pass that image to the MFMailComposeViewController and e-mail it, it arrives at the destination E-Mail address and is displayed there incorrectly in landscape mode.

However, if I shoot the picture in landscape mode and then send it, it is displayed at the E-mail's destination correctly in landscape mode

So, how can I make a picture shot in portrait mode arrive at the E-Mail destination in portrait mode? That was my original question.

Here's the code, as I showed it in the original question, except that I am now sending the image as a JPEG rather than a PNG but this has made no difference.

This is how I'm capturing the image with the imagePickerController and placing it into a global called gImag:

gImag = (UIImage *)[info valueForKey: UIImagePickerControllerOriginalImage];
[[self imageView] setImage: gImag];             // send image to screen
[self imagePickerControllerRelease: picker ];   // free the picker object

And this is how I E-Mail it using the MFMailComposeViewController:

if ( [MFMailComposeViewController canSendMail] )
   {
   MFMailComposeViewController * mailVC = [MFMailComposeViewController new];
   NSArray * aAddr  = [NSArray arrayWithObjects: gAddr, nil];
   NSData * imageAsNSData = UIImageJPEGRepresentation( gImag );

   [mailVC setMailComposeDelegate: self];
   [mailVC        setToRecipients: aAddr];
   [mailVC             setSubject: gSubj];
   [mailVC      addAttachmentData: imageAsNSData
                         mimeType: @"image/jpg"
                         fileName: @"myPhoto.jpg"];
   [mailVC         setMessageBody: @"Blah blah"
                           isHTML: NO];

   [self presentViewController: mailVC
                      animated: YES
                    completion: nil];
   }
else NSLog( @"Device is unable to send email in its current state." );

When I describe how to solve this, I’m going to focus on working with an iPhone 5 and using its main camera which shoots at 3264x2448 just to keep things simple. This problem does, however, affect other devices and resolutions.

The key to unlocking this problem is to realize that when you shoot an image, in either portrait of landscape, the iPhone always stores the UIImage the same way: as 3264 wide and 2448 high.

A UIImage has a property which describes its orientation at the time the image was captured and you can get it like this:

UIImageOrientation orient = image.imageOrientation;

Note that the orientation property does not describe how the data in the UIImage is physically configured (as as 3264w x 2448h); it only describes its orientation at the time the image was captured.

The property’s use is to tell software, which is about to display the image, how to rotate it so it will appear correctly.

If you capture an image in portrait mode, image.imageOrientation will return UIImageOrientationRight. This tells displaying software that it needs to rotate the image 90 degrees so it will display correctly. Be clear that this ‘rotation’ does not effect the underlying storage of the UIImage which remains as 3264w x 2448h.

If you capture an image in landscape mode, image.imageOrientation will return UIImageOrientationUp. UIImageOrientationUp tells displaying software that the image is fine to display as it is; no rotation is necessary. Again, the underlying storage of the UIIMage is 3264w x 2448h.

Once you are clear about the difference between how the data is physically stored vs. how the orientation property is used to describe its orientation at the time it was captured, things begin to get clearer.

I created a few lines of debugging code to ‘see’ all of this.

Here’s the imagePickerController code again with the debugging code added:

gImag = (PIMG)[info valueForKey: UIImagePickerControllerOriginalImage];

UIImageOrientation orient = gImag.imageOrientation;
CGFloat            width  = CGImageGetWidth(gImag.CGImage);
CGFloat            height = CGImageGetHeight(gImag.CGImage);

[[self imageView] setImage: gImag];                 // send image to screen
[self imagePickerControllerRelease: picker ];   // free the picker object

If we shoot portrait, gImage arrives with UIImageOrientationRight and width = 3264 and height = 2448.

If we shoot landscape, gImage arrives with UIImageOrientationUp and width = 3264 and height = 2448.

If we continue on to the E-Mailng MFMailComposeViewController code, I’ve added the debugging code in there as well:

if ( [MFMailComposeViewController canSendMail] )
   {
   MFMailComposeViewController * mailVC = [MFMailComposeViewController new];
   NSArray * aAddr  = [NSArray arrayWithObjects: gAddr, nil];

   UIImageOrientation orient = gImag.imageOrientation;
   CGFloat            width  = CGImageGetWidth(gImag.CGImage);
   CGFloat            height = CGImageGetHeight(gImag.CGImage);

   NSData * imageAsNSData = UIImageJPEGRepresentation( gImag );

   [mailVC setMailComposeDelegate: self];
   [mailVC        setToRecipients: aAddr];
   [mailVC             setSubject: gSubj];
   [mailVC      addAttachmentData: imageAsNSData
                         mimeType: @"image/jpg"
                         fileName: @"myPhoto.jpg"];
   [mailVC         setMessageBody: @"Blah blah"
                           isHTML: NO];

   [self presentViewController: mailVC
                      animated: YES
                    completion: nil];
   }
else NSLog( @"Device is unable to send email in its current state." );

Nothing amazing to see here. We get exactly the same values as we did back in the imagePickerController code.

Let's see exactly how the problem manifests:

To begin, the camera takes a portrait shot and it is displayed correctly in portrait mode by the line:

[[self imageView] setImage: gImag];                 // send image to screen

It is display correctly because this line of code sees the orientation property and rotates the image appropriately (while NOT touching the underlying storage at 3264x2448).

Flow-of-control goes to the E-Mailer code now and the orientation property is still present in gImag so when the MFMailComposeViewController code displays the image in the outgoing E-Mail, it is correctly oriented. The physical image is still stored as 3264x2448.

The E-Mail is sent and, on the receiving end, knowledge of the orientation property has been lost so the receiving software displays the image as it is physically laid out as 3264x2448, i.e. landscape.

In debugging this, I ran into an additional confusion. And that is that the orientation property can be stripped from the UIImage, if you make a copy of it incorrectly.

This code shows the problem:

if ( [MFMailComposeViewController canSendMail] )
   {
   MFMailComposeViewController * mailVC = [MFMailComposeViewController new];
   NSArray * aAddr  = [NSArray arrayWithObjects: gAddr, nil];

   UIImageOrientation orient = gImag.imageOrientation;
   CGFloat            width  = CGImageGetWidth(gImag.CGImage);
   CGFloat            height = CGImageGetHeight(gImag.CGImage);

   UIImage * tmp = [[UIImage alloc] initWithCGImage: gImag.CGImage];

   orient = tmp.imageOrientation;
   width  = CGImageGetWidth(tmp.CGImage);
   height = CGImageGetHeight(tmp.CGImage);

   NSData * imageAsNSData = UIImageJPEGRepresentation( tmp );

   [mailVC setMailComposeDelegate: self];
   [mailVC        setToRecipients: aAddr];
   [mailVC             setSubject: gSubj];
   [mailVC      addAttachmentData: imageAsNSData
                         mimeType: @"image/jpg"
                         fileName: @"myPhoto.jpg"];
   [mailVC         setMessageBody: @"Blah blah"
                           isHTML: NO];

   [self presentViewController: mailVC
                      animated: YES
                    completion: nil];
   }
else NSLog( @"Device is unable to send email in its current state." );

When we look at the debugging data for the new UIImage tmp, we get UIImageOrientationUp and width = 3264 and height = 2448.

The orientation property was stripped and the default orientation is Up. If you do not know the stripping is going on, it can really confuse things.

If I run this code, I now get the following results:

Things are unchanged in the imagePickerController code; the image is captured as before.

Flow-of-control goes on to the E-Mailer code but now the orientation property has been stripped from the tmp image so when the MFMailComposeViewController code displays the tmp image in the outgoing E-Mail, it is shown in landscape mode (because the default orientation is UIImageOrientationUp so there is no rotation done of the 3264x2448 image).

The E-Mail is sent and on the receiving end, knowledge of the orientation property is missing as well so the receiving software displays the image as it is physically laid out as 3264x2448, i.e. landscape.

This ‘stripping’ of the orientation property when making copies of the UIImage can be avoided, if one is aware it is going on, by using the following code to make the UIImage copy:

UIImage * tmp = [UIImage imageWithCGImage: gImag.CGImage
                                    scale: gImag.scale
                              orientation: gImag.imageOrientation];

That would allow you to avoid losing the orientation property along the way but it would still not deal with its loss on the far end when you e-mail the image.

There’s a better way than all this messing about and worrying about the orientation property.

I found some code here that that I’ve integrated into my program. This code will physically rotate the underlying stored image according to its orientation property.

For a UIImage with an orientation of UIImageOrientationRight, it will physically rotate the UIImage so it ends up as 2448x3264 and it will strip away the orientation property so it is seen thereafter as the default UIImageOrientationUp.

For an UIImage with an orientation of UIImageOrientationUp, it does nothing. It lets the sleeping landscape dogs lie.

If you do this then I think (based on what I’ve seen so far), that the orientation property of the UIImage is superfluous thereafter. So long as it remains missing/stripped or set to UIImageOrientationUp, your images should be displayed correctly at each step along the way and on the distant end when the images embedded in your E-Mail is displayed.

Everything I’ve discussed in the answer, I have personally single-stepped and watched it happen.

So, here my final code that works:

gImag = (PIMG)[info valueForKey: UIImagePickerControllerOriginalImage];
[[self imageView] setImage: gImag];             // send image to screen
[self imagePickerControllerRelease: picker ];   // free the picker object

and

if ( [MFMailComposeViewController canSendMail] )
   {
   MFMailComposeViewController * mailVC = [MFMailComposeViewController new];
   NSArray                     * aAddr  = [NSArray arrayWithObjects: gAddr, nil];

   //...lets not touch the original UIImage

   UIImage * tmpImag = [UIImage imageWithCGImage: gImag.CGImage
                                           scale: gImag.scale
                                     orientation: gImag.imageOrientation];

   //...do physical rotation, if needed

   PIMG ImgOut = [gU scaleAndRotateImage: tmpImag];

   //...note orientation is UIImageOrientationUp now

   NSData * imageAsNSData = UIImageJPEGRepresentation( ImgOut, 0.9f );

   [mailVC setMailComposeDelegate: self];
   [mailVC        setToRecipients: aAddr];
   [mailVC             setSubject: gSubj];
   [mailVC      addAttachmentData: imageAsNSData
                         mimeType: @"image/jpg"
                         fileName: @"myPhoto.jpg"];
   [mailVC         setMessageBody: @"Blah blah"
                           isHTML: NO];

   [self presentViewController: mailVC
                      animated: YES
                    completion: nil];
   }
else NSLog( @"Device is unable to send email in its current state." );   

And, finally, here's the code I grabbed from here that does the physical rotation, if necessary:

- (UIImage *) scaleAndRotateImage: (UIImage *) imageIn
   //...thx: http://blog.logichigh.com/2008/06/05/uiimage-fix/
   {
   int kMaxResolution = 3264; // Or whatever

   CGImageRef        imgRef    = imageIn.CGImage;
   CGFloat           width     = CGImageGetWidth(imgRef);
   CGFloat           height    = CGImageGetHeight(imgRef);
   CGAffineTransform transform = CGAffineTransformIdentity;
   CGRect            bounds    = CGRectMake( 0, 0, width, height );

   if ( width > kMaxResolution || height > kMaxResolution )
      {
      CGFloat ratio = width/height;

      if (ratio > 1)
         {
         bounds.size.width  = kMaxResolution;
         bounds.size.height = bounds.size.width / ratio;
         }
      else
         {
         bounds.size.height = kMaxResolution;
         bounds.size.width  = bounds.size.height * ratio;
         }
      }

   CGFloat            scaleRatio   = bounds.size.width / width;
   CGSize             imageSize    = CGSizeMake( CGImageGetWidth(imgRef),         CGImageGetHeight(imgRef) );
   UIImageOrientation orient       = imageIn.imageOrientation;
   CGFloat            boundHeight;

   switch(orient)
      {
      case UIImageOrientationUp:                                        //EXIF = 1
         transform = CGAffineTransformIdentity;
         break;

      case UIImageOrientationUpMirrored:                                //EXIF = 2
         transform = CGAffineTransformMakeTranslation(imageSize.width, 0.0);
         transform = CGAffineTransformScale(transform, -1.0, 1.0);
         break;

      case UIImageOrientationDown:                                      //EXIF = 3
         transform = CGAffineTransformMakeTranslation(imageSize.width, imageSize.height);
         transform = CGAffineTransformRotate(transform, M_PI);
         break;

      case UIImageOrientationDownMirrored:                              //EXIF = 4
         transform = CGAffineTransformMakeTranslation(0.0, imageSize.height);
         transform = CGAffineTransformScale(transform, 1.0, -1.0);
         break;

      case UIImageOrientationLeftMirrored:                              //EXIF = 5
         boundHeight = bounds.size.height;
         bounds.size.height = bounds.size.width;
         bounds.size.width = boundHeight;
         transform = CGAffineTransformMakeTranslation(imageSize.height, imageSize.width);
         transform = CGAffineTransformScale(transform, -1.0, 1.0);
         transform = CGAffineTransformRotate(transform, 3.0 * M_PI / 2.0);
         break;

      case UIImageOrientationLeft:                                      //EXIF = 6
         boundHeight = bounds.size.height;
         bounds.size.height = bounds.size.width;
         bounds.size.width = boundHeight;
         transform = CGAffineTransformMakeTranslation(0.0, imageSize.width);
         transform = CGAffineTransformRotate(transform, 3.0 * M_PI / 2.0);
         break;

      case UIImageOrientationRightMirrored:                             //EXIF = 7
         boundHeight = bounds.size.height;
         bounds.size.height = bounds.size.width;
         bounds.size.width = boundHeight;
         transform = CGAffineTransformMakeScale(-1.0, 1.0);
         transform = CGAffineTransformRotate(transform, M_PI / 2.0);
         break;

      case UIImageOrientationRight:                                     //EXIF = 8
         boundHeight = bounds.size.height;
         bounds.size.height = bounds.size.width;
         bounds.size.width = boundHeight;
         transform = CGAffineTransformMakeTranslation(imageSize.height, 0.0);
         transform = CGAffineTransformRotate(transform, M_PI / 2.0);
         break;

      default:
         [NSException raise: NSInternalInconsistencyException
                     format: @"Invalid image orientation"];

      }

   UIGraphicsBeginImageContext( bounds.size );

   CGContextRef context = UIGraphicsGetCurrentContext();

   if ( orient == UIImageOrientationRight || orient == UIImageOrientationLeft )
      {
      CGContextScaleCTM(context, -scaleRatio, scaleRatio);
      CGContextTranslateCTM(context, -height, 0);
      }
   else
      {
      CGContextScaleCTM(context, scaleRatio, -scaleRatio);
      CGContextTranslateCTM(context, 0, -height);
      }

   CGContextConcatCTM( context, transform );

   CGContextDrawImage( UIGraphicsGetCurrentContext(), CGRectMake( 0, 0, width, height ), imgRef );
   UIImage *imageCopy = UIGraphicsGetImageFromCurrentImageContext();
   UIGraphicsEndImageContext();

   return( imageCopy );
   }

Cheers, from New Zealand.

Answer 6

I think you can get the UIImage API to handle the rotation for you automatically, without needing to manually do the transforms.

The documentation for the UIImage methods size and drawInRect: both say they take the orientation into account, so I drawInRect the UIImage into a new context and grab the resultant (auto-rotated) image from there. Here's the code in a category:

@interface UIImage (Orientation)

- (UIImage*)imageAdjustedForOrientation;

@end

@implementation UIImage (Orientation)
- (UIImage*)imageAdjustedForOrientation
{
    // The UIImage methods size and drawInRect take into account 
    //  the value of its imageOrientation property
    //  so the rendered image is rotated as necessary.
    UIGraphicsBeginImageContextWithOptions(self.size, NO, self.scale);

    [self drawInRect:CGRectMake(0, 0, self.size.width, self.size.height)];

    UIImage *orientedImage = UIGraphicsGetImageFromCurrentImageContext();

    UIGraphicsEndImageContext();

    return orientedImage;
}
@end

I just found this better answer by an0 which is virtually the same, but also includes the optimisation of not re-redrawing the image if the orientation is already correct.