No, usually, you will get an email from a server saying it has not been able to deliver the message during the last 24 (or 12, or 8 or whatever) hours.
This is a courtesy to you, just to let you know that the person has not received the email. However, it usually also says that it will continue trying to deliver for the next 72 hours or so.
Many times, during that period, the destination server is brought back up to accept your email, and delivery takes place at that time.
Sawedoff