This isn't really something that you do in git. Remember, you're using a DVCS, so you're pulling down entire repository history - what is supposed to happen to the commits that change files outside the directory you're pulling down?
If a project is sufficiently large, then the maintainer of whatever repo you're cloning should use submodules to divide up the sections, so that developers can only pull down the areas they want to work on.
Git (well, DVCSs in general) is not designed for endusers so much as it is for developers. I find git (especially in combination with github) to be incredibly helpful for coding. If there's a project on github that I want to contribute to, I just fork it, pull it down, hackhackhack, push up, and send a pull request, easypeasy, rather than making a checkout, hacking, creating a diff, trying to find where to email it to, and hoping that the diff doesn't break before the developers get around to merging it in.
Git as a whole is certainly more complicated than svn, but that's because coding is a complicated process. The centralized VCS workflow is incredibly painful for me, because I was raised on DVCSs, and I'm not allowed to do all sorts of things that I'm used to doing. So, yes, it will be more complicated to use git vs. svn if you do the same things, but the advantage is that there are so many more things that you can do. Git was developed for linux kernel development, and it shows.