I would suggest taking a look at the source code to Smultron. It has very nice syntax highlighting. It uses a subclass of NSTextView to do most of the heavy lifting. The code uses the layout manager to add attributes to the text and uses some other clever tricks to only highlight as much of the document as necessary.
