Have you got a lot of formatting in the Report, i.e. highlighting, alternating font colours, formulae, graphics, charts etc..
All these are factors as to how quickly the report will run, even though the data may be being returned quickly, if there is a lot of report rendering too then this...