I\'ve been using javamail to retrieve mails from IMAP server (currently GMail). Javamail retrieves list of messages (only ids) in a particular folder from server very fast,
I'm running into the same thing. After profiling, I noticed that getBody was being called every time I tried to do a message.getFrom() like you are, even though I was only accessing fields that should be covered by the Envelope flag. See https://java.net/projects/javamail/forums/forum/topics/107956-gimap-efficiency-when-only-reading-headers