No idea where the problem lies. You might want to test where the delay is caused. That should sort out the delays.
1) look at the sender, is it going out of the outbox?
2) look at the server, is it processing on time?
3) look at the recipient, is he/she connected to the server?