I found that links:
The prerequisites have changed since .NET 3.5 SP1, .NET 4.0 requires at least XP SP3, Windows 2003 SP2 or Vista SP1. .NET 3.5 could be installed on any version of these operating systems, including a 11 year old completely unpatched version of XP.
So the 3.5 installer also needs to be able to update a bunch of native Windows components.
Not having to include the Windows component updates and additions allowed for a significant size reduction. If that's an issue then you'll have to include the Windows service pack installers on your pen drive.