You sure can (for example, FileOutputStream does).
Throwing an exception from the constructor should be done wisely - make sure you clean up after yourself cleanly. Throwing exceptions in a constructor is sometimes done in order to make sure RAII is held.