Drupal is great, but it is very inefficient. But, as I keep telling my frontend developers, this is the payoff for being flexible and highly modular.
We are currently creating a Druapl site, which I consider enterprise (http://stackoverflow.com/questions/161991/what-makes-an-application-an-enterprise-or-enterprise-level-application). It has about 40 content types, 15 user roles, integrated with Ubercart, interfacing with external APIs for membership verification etc. etc. Our development servers struggle some times, but on our staging hosting, it is lightening - fast as any similar sized site.
Make sure any bespoke code is well developed, use Drupal's performance features before release and ensure quality hosting is used.