I am creating a syntax highlighter for the iPhone and in order to display text with multiple formats, I have sub-classed UIView and modified the drawRect:
method so
You should first spend some quality time understanding how this problem was solved on Mac:
http://developer.apple.com/documentation/Cocoa/Conceptual/TextArchitecture/Tasks/AssembleSysByHand.html
http://developer.apple.com/documentation/Cocoa/Conceptual/TextLayout/TextLayout.html
In particular, you should become familiar with line fragment generation, which is the problem you're trying to solve for word-wrap, and you should understand glyph generation in order to do rich text well. You don't need to implement all of the flexibility of NSTypesetter or NSLayoutManager of course, but the text system on Mac is incredibly powerful, and you should learn from its example. Implementing something similar to NSAttributedString for iPhone may be valuable to you and improve your performance.
Unless you're moving things around a lot (for which this would best work in a UIScrollView), I don't think you should need to use CALayers here. That seems overkill for the problem, and may actually adversely impact the optimizations already provided by UIScrollView. If you're seeing performance problems, first make sure you're not doing redundant calculations within your drawRect:.