Yes, quality may vary slightly depending on the preset used, but it should not be a significant amount. Here's an excerpt from a discussion on #x264. Similar question to yours with answers provided by one of the x264 developers:
verb3k | Do different presets have an effect on quality when used with CRF?
@Dark_Shikari | verb3k: yes, but not too much.
@Dark_Shikari | a 0th-order approximation is that they have no effect.
@Dark_Shikari | The main reason there's a difference is because the preset affects how x264 itself measures quality
@Dark_Shikari | that is, it uses better, more accurate methods of measuring quality
@Dark_Shikari | obviously, this will affect the definition of what -crf does!
@Dark_Shikari | It's just not too much, so we can mostly ignore it.
@Dark_Shikari | specifically, there are three big things that can affect the definition of quality
@Dark_Shikari | 1) AQ being on/off
@Dark_Shikari | jump: ultrafast to superfast
@Dark_Shikari | 2) mbtree being on/off
@Dark_Shikari | jump: superfast to veryfast
@Dark_Shikari | 3) psy-rd being on/off
@Dark_Shikari | jump: faster to fast
@Dark_Shikari | above fast there are no more big jumps.
This means that a slower preset with the same CRF value will improve quality per bitrate, but might make both quality and bitrate higher or lower.
