How to create Apple mail plugin

Question

I\'m going to create a mail plugin for the OS X Mail.app application for some additional features.

I have no idea where to start as there is no official documentatio

Answer 1

There is no official supported way to build such a tool - you need to start trying to hook in to Mail.app without any official support.

If you want to persist on this sort of thing, then you'll need to understand how Mail.app internals work, which is a bunch of using the debugger and class dump to inspect libraries in other apps:

https://github.com/nygard/class-dump

You'll probably also want a way to inject code into other applications, for example:

https://github.com/rentzsch/mach_inject

And every time Apple update Mail.app you'll potentially need to redo everything :)

Answer 2

As noted, writing Apple Mail plugins is not straightforward, since it only has a private plugin API, which is entirely undocumented and can change with any new version of Mail.app. The best code example is GPGMail, which is open source & still active (already working on Yosemite support). Here is what I successfully did to get started (will put it up on github once finished):

How to build a minimal Apple Mail plugin (as of Mavericks & Xcode 6.0.1)

1. you need to create an OSX "Bundle" project in XCode
2. wrapper extension is mailbundle (under Packaging in the project Build settings)
3. a bundle needs to be stored under ~/Library/Mail/Bundles (as Build Phase add a Copy Files action with that as absolute path destination and the *.mailbundle from your build/ folder as item to copy)
4. for development, I have set up /Applications/Mail.app as executable in my run scheme, so that Run in XCode will build it, copy the bundle and start mail; note that at this point you'll get an error from Mail that your plugin cannot be started and was disabled
5. you need to provide a list of SupportedPluginCompatibilityUUIDs in the Info.plist, I stole it from GPGMail, these change with new Mail/OSX versions
6. use class-dump to generate the header files from Mail.app's private API
7. starting point is MVMailBundle, which you have to inherit from and which has a registerBundle method to hook you in
   • I extracted that from the huge generated header file in a small MVMailBundle.h header to include where needed (as done by GPGMail)
8. create a new class MyMailBundle, inheriting from NSObject
   • it needs an initialize method
   • and set it as "Principle class" in the Info.plist so that it gets run when the bundle is loaded by Mail.app

#import <Cocoa/Cocoa.h>

@interface MyMailBundle : NSObject

+ (void)initialize;
@end

9. initialize implementation: previously, you could use the simple way and directly inherit as done in Letterbox, however, since 64-bit runtimes of Objective-C you have to use the dynamic way as done by GPGMail:
   • using NSClassFromString to dynamically get the MVMailBundle class
   • and class_setSuperclass from <objc/runtime.h> to have your own class inherit from it
   • and then call registerBundle on it casted as MVMailBundle (requires include of MVMailBundle.h)

#import <objc/runtime.h>
#import "MVMailBundle.h"
#import "MyMailBundle.h"

@implementation MyMailBundle

+ (void)initialize
{
    NSLog(@"Loading MyMail plugin...");

    // since 64-bit objective-c runtimes, you apparently can't load
    // symbols directly (i.e. through class inheritance) and have to
    // resort to NSClassFromString
    Class mvMailBundleClass = NSClassFromString(@"MVMailBundle");

    // If this class is not available that means Mail.app
    // doesn't allow plugins anymore or has changed the API
    if (!mvMailBundleClass)
        return;

    // dynamically change super class hierarchy
#pragma GCC diagnostic push
#pragma GCC diagnostic ignored "-Wdeprecated"
    class_setSuperclass([self class], mvMailBundleClass);
#pragma GCC diagnostic pop

    // register our plugin bundle in mail
    [[((MyMailBundle *)self) class] registerBundle];

    NSLog(@"Done registering MyMail plugin.");
}
@end

10. add some NSLog logging calls to verify the right thing is happening, they'll be visible in XCode's console when running/debugging Mail.app from within XCode or alternatively in the system logs of Console.app
11. This should successfully run the plugin in Mail with no error!
12. The next steps involve crazy things like MethodSwizzling and ClassPosing to modify Mail's behavior, where GPGMail can be a helpful example. (Haven't been there myself yet)

For reference, here are some of the resources that helped me:

• GPGMail
• Adam Nash: Getting Ready to Write an Apple Mail.app Plug-in for Mac OS X - some good links, but apparently he never finished the project, so no code
• James R. Eagan: Demystifying Mail.app Plugins on Leopard - using PyObjC to write a plugin in Python, explains the basic mechansims, very useful
• Aaron Harnly: Mail Plugin Template - for XCode 2 I think, unfortunately the template (download a zip) doesn't work as template in Xcode anymore, but the code is still useful to look at
• Aaron Harnly: Letterbox sources - from the same guy, but also from 2007, very outdated; contains a readme from the template, though it doesn't really help if you can't use the template.