Is the report doing it's own calculations on the data that is returned which could potentially take time and resources to accomplish?
Is the report drawing graphs that have lots and lots of datapoints which may take longer to reproduce than a graph with a larger increment?
Is the report loading images? How are the images handled? Are they embedded or linked from an outside source?
Are you creating a lot of groups or drill downs which could take time on the report loading to create?
Answering these questions may start to point you in the correct direction as far as how to make your report more efficient.
