uBlas, a part of Boost. It offers full BLAS level 1-3, and hence lots of array arithmetic functions.
Armadillo also seems to be a C++ linear algebra library, which as far as I can see optionally uses LAPACK/Atlas (which of course makes it canonically fast).
The GNU Scientific Library offers full BLAS. I don't know how fast it is, or if it can use LAPACK/Atlas.
If you don't need anything more fancy than what you list, you can quite easily wrap for example Atlas' BLAS yourself. But you probably don't want to reinvent the wheel if you don't have to.