I have a psychological tic which makes me reluctant to use large libraries (like GLib or Boost) in lower-level languages like C and C++. In my mind, I think:
There may be a small overhead when loading these libraries if they're dynamically linked. This will typically be a tiny, tiny fraction of the time your program spends running.
However there will be no overhead once everything is loaded.
If you don't want to use all of boost, then don't. It's modular, so you can use the parts you want and ignore the rest.