Web site cannot be accessed

[Mike Lewis]

I hope this is the right forum for this question.

Over the last couple of months, I've noticed that my company's website is sometimes inaccessible. In particular, when I try to visit it, I see a "server not found" message in Firefox, or "Cannot find server or DNS Error" in IE. When I try to ping the site, I see "could not find host ...".

This is very intermittent. It typically happens five or six times a week, but sometimes more. Sometimes the outage only lasts a few minutes, but often it lasts two or three hours.

For the last three weeks, the web hosting company has been monitoring the site, but they say they have not seen any outages. I've also signed up with a web monitoring service (SiteUptime.Com). This checks the site every 30 minutes. They too are reporting 100 percent uptime.

So my question is: How is it possible that I can't access the site, but these monitoring services can?

I'm as sure as I can be that there is nothing wrong with my network or internet connection. I have no difficulty visiting other sites, and I have triple checked the URLs in my bookmarks.

Also, is there any way I can find out for sure if other people can access the site? I can monitor the site from my own PC, but I don't have access to another system that would allow me to check it independently.

Thanks in advance for any help.

Mike


__________________________________
Mike Lewis (Edinburgh, Scotland)

My Visual FoxPro site: www.ml-consult.co.uk

 

[pansophic]

It sounds as if your DNS server may be timing out under load. You could install Ethereal and check the to see if that is what is happening.

How many DNS servers do you have listed in your DNS configuration? You should have at least 2, preferably 3, and at least two of them should be from independent service providers.

If you want to check your own service availability, it is probably desirable to use the DNS server specified in your SOA record.

Most DNS servers cache records that have been recently accessed, but may timeout if they are under heavy load, or your link to the ISP is under heavy load. DNS access is UDP typically and is therefore first on the list to be dropped during congestion.


pansophic

 

[Mike Lewis]

Pansophic,

Many thanks for your reply, which I read with interest. I also took a look at the Ethereal site. Unfortunately, all this is way beyond my expertise.

You ask how many DNS servers I have listed in my DNS configuration. I don't know where to start looking for this. Is the DNS configuration something to do with my web host, or my ISP?

And what is an SOA record? Where do I look for it.

I'd be grateful for any further help you can give. Thanks for your patience.

Mike


__________________________________
Mike Lewis (Edinburgh, Scotland)

My Visual FoxPro site: www.ml-consult.co.uk

 

[pansophic]

Mike, thought that I had commented already, but apparently I never actually posted the response.

Ethereal isn't too bad once you open it. It color codes all of the protocols, so it becomes pretty easy to see the DNS requests and responses.

If you open a console (DOS) and type 'ipconfig /all' it will give you a detailed listing of how your networking is configured. Look for the line 'DNS Servers' and if there is only one IP address, then you only have one DNS server configured. You are probably getting your DNS configuration from your cable modem (or other DHCP server), so you'd have to make changes there.

The SOA record is your Start of Authority record for your domain. It is the Authoritative (final) authority for your domain and contains all of the records for your domain. When your computer requests that a name be resolved, it sends a request to your DNS server. That server is probably a caching server, so it looks up the name in its cache. If the name is there, it serves the IP address. Otherwise it sends a request to the DNS server in its configuration. That server will probably request the SOA from one of the root DNS servers, which will provide the DNS server(s) IP address(es). Then that server will request the DNS entry from a server identified in the SOA. That will be passed down from server to server until it gets to your machine. In the mean time, your system may have timed out on its DNS request, which is the errors that you are seeing.

If you point to the DNS server in your SOA as your secondary DNS server, you should always be able to resolve to your web site.


pansophic

 

[Mike Lewis]

Pansophic,

Thanks again. Your reply was very helpful and informative.

However, I'm not sure if it answers my basic question. My problem is not that I want to access my web site when it's down. My problem is that I have no way of knowing if other people in the world can access it when I can't.

There are often times when I can't get at the site, but the hosting company and monitoring service both report that the site is up. If I can be sure that there is a problem at my end (in my PC or router, or at my ISP, for example), then that's all I need to know. But I can't see how that can be the case, given that I never have any particular problems accessing other sites.

Mike




__________________________________
Mike Lewis (Edinburgh, Scotland)

My Visual FoxPro site: www.ml-consult.co.uk

 

[pansophic]

In all probability your website is not down, because the errors that you are reporting are DNS errors, not connection errors. You are never resolving the name to an IP, and therefore cannot connect to your site.

You have two options, one is to add a second and/or third DNS server to your configuration (one of the DNS servers listed in your SOA would be ideal), theoretically ensuring that your name actually gets resolved, or by attaching to the site by IP address when you get this type of failure (you would probably have to do this manually since the hosting service probably uses virtual hosts).

To connect manually to your site, you'll need the IP address. There are many sites that can do a lookup for you, like:

http://www.dnsstuff.com/.

You'll find the host lookup in the upper right-hand corner.

In a DOS prompt type:

c:\> telnet 192.168.73.2 80<enter>
GET / HTTP/1.1<enter>
Host:[URL unfurl="true"]www.mydomain.com<enter>[/URL]
<enter>

This will dump the index (or default) page to your screen. If it works by IP address only, then you have a DNS problem.

One of the big issues with DNS problems is that they are very transient, and there are many reasons for them to occur. Many times they will occur because an upstream DNS server is very busy, or because one of the links that you are traversing is very busy. Because this is possible, failover was designed into the DNS system from the very beginning, with the use of two or three (or more) servers so that a client could almost certainly be guaranteed of getting a response, even if a transient error occurs.


pansophic

 

[Mike Lewis]

Panasophic,

What I've decided to do is to keep checking the site at regular intervals during the day (I might even write a Visual FoxPro program to automate that). Then, the next time it appears to be down, I'll use a service such as

http://www.Anonymizer.com

to connect to the site.

If I can't access the site, but Anonymizer can, then I'll assume it's a DNS-resolution issue, and my problem will be solved. If neither can access the site, then I'll know it's the web hosting that's at fault, and will take the appropriate action.

The only problem is that the outages occur fairly infrequently, so it's difficult to do this testing in a controlled way.

Anyway, many thanks for your help and for your informative replies.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

My Visual FoxPro site: www.ml-consult.co.uk

 

[stooo]

You could always get another company to monitor the website for you. A quick google for website monitoring shows a lot of fairly cheap options.

Stu

 

[Mike Lewis]

Stu,

Thanks for that. In fact, I signed up with SiteUptime.Com when the problem first arose.

Siteuptime was reporting 100% uptime, even though I was seeing downtime. It was that scenario that prompted me to start this thread.

Right now, I have my home-grown monitoring sysem in place. I'll see what information that throws up.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

My Visual FoxPro site: www.ml-consult.co.uk

 

[pansophic]

Mike,

I don't know why I didn't think of it before.

You can use wget to check the site by IP address and get the correct page. If you use 'wget --header="Host:

http://www.mydomain.com"

-a test.log 192.168.84.3' it will get your index page off of the host. It will automatically save that as a file. If you use 'wget

http://www.mydomain.com

-a test.log' it will look the host up by name. Anytime the IP address works and the name lookup doesn't, you can be assured that it is a DNS issue. The log file 'test.log' will contain entries for each attempt, so you can search for the failed DNS entries quickly, or get a count by grepping the log and piping to wc.

wget is a *nix package, but it is bundled with Cygwin (a free download) which allows many Gnu apps to run under Windows.

You could script this using WSH or as a batch file using AT.


pansophic

 

[Mike Lewis]

Pansophic,

Sounds like a good idea. I'll give that a try. Thanks.

Mike


__________________________________
Mike Lewis (Edinburgh, Scotland)

My Visual FoxPro site: www.ml-consult.co.uk