DNS vs. SMTPROUTES?

[cejohnsonsr]

In a previous post I worked through getting mail to leave my local network. The solution was to enter e relayhost in smtproutes. That is really not what I want to have to do. I would much prefer a DNS lookup to determine the routing. So my question is: why isn't a DNS lookup being done? Is there some file where I need to identify a DNS server? Or do I need to be running my own DNS server for lookups?

Ed

 

[Noway2]

You should not have to run your own DNS server for mail, only to have a working DNS server. The main reason for needing to relay through another SMTP host is because your server is not recognized as being a valid server.

On three occasions in the other thread I have asked questions and asked you to perform checks that would determine if this was the case of your problem, you have not answered the questions. I also tried to explain to you the how and why this is important. Based on what you have indicated so far, I highly suspect that it is your problem.

 

[cejohnsonsr]

Well 1st, let me apologize for not answering your questions directly & fully. I've had too much on my plate lately to be able to concentrate properly on anything.

Now.....

>When you tried to connect to yahoo via telnet, where did >it fail and what error message did you get?

It never connected. The connection request was refused.

>Go to this site:

http://whatismyipaddress.com/ip-lookup

>and do a lookup of your IP. Does it tell you that your IP >is static or dynamic?

It's a static IP & I'm not on any blacklist. (Off topic sort of: I don't see what difference a static vs dynamic IP would make or even how the web app knows. I'm actually pretty sure it's really a dynamic IP because my ISP charges extra for a static.)

>If it does NOT say static, most ISPs will NOT accept your >mail.

The whole point is that I don't want to ask my ISP to accept my mail. I want it to be routed properly to the appropriate mail server, based on the destination of any given email. However, I DID get mail out when I specified my ISP's mail server as a static route for the 1st hop.

>What address are you using to try and telnet into yahoo? >I tried yahoo.com on port 25 and received nothing. If you >use smtp.mail.yahoo.com you will get an "530 >authentication required - for help go to >

http://help.yahoo.com/help/us/mail/"

notice when you try.

I used mail.yahoo.com & smtp.yahoo.com on port 25. The response was an attempt to connect to several (3 I think) mail servers in the order they were discovered/presented. In the end the connection attempt was refused.

>This brings up another possibility: does your email server >have access to a proper DNS that will obtain the MX >records (ie, the mail servers)?

That was MY question, Dude.

I know I have access to DNS in general because I'm here. What I'm currently trying to determine is if I've missed a configuration setting (hence the subject of this post & the end of the last (the answer to which I never was able to ascertain.))

Here's a log entry from qmail-send:

@400000004c99446402bbe1ec status: local 0/10 remote 0/20
@400000004c9944af0704f964 starting delivery 11: msg 8920413 to remote c_johnsonsr@yahoo.com
@400000004c9944af07050134 status: local 0/10 remote 1/20
@400000004c99459612d69be4 delivery 11: deferral: Sorry,_I_wasn't_able_to_establish_an_SMTP_connection.(#4.4.1)/

>What address is it trying to connect to when it makes a >connection to yahoo?

I don't know. Telnet responded with the names of the servers as well as the addresses (which would indicate that DNS is working via telnet), but I didn't bother to write it down. I figured the connection refusal was due to ident/auth which I wasn't able to properly provide in the connection request.

I should also mention a couple of other things that are only sort of on topic:

1st, please remember that I haven't tried this in about 6 or 7 years. My skills are not JUST a little rusty.

2nd, the original post concerning error 4.4.1 led me to believe there was a problem in the communication between the servers. As you pointed out previously, the error message is pretty vague. It gives no indication as to WHY the connection couldn't be established. A lot of troubleshooting later, I've pretty much narrowed it down to a failure to obtain a DNS lookup. Here's another entry from the qmail-send log:

@400000004c99431e2953fe84 new msg 8920413
@400000004c99431e29540a3c info msg 8920413: bytes 236 from <papa@******.****.***> qp 13322 uid 1000
@400000004c99431e29f92bac starting delivery 7: msg 8920413 to remote c_johnsonsr@yahoo.com
@400000004c99431e29f93f34 status: local 0/10 remote 1/20
@400000004c9943332bd6e0a4 delivery 7: deferral: CNAME_lookup_failed_temporarily._(#4.4.3)/

This error is a little better, although I don't know why DNS would be doing a CNAME lookup to obtain an MX record. I would think there should be an A record lookup, since the A record points to the MX record. At any rate, this is the only entry that indicated a DNS problem, so I went with it. I gave the server a static route & BINGO! Outbound mail works. Now I'm trying to figure out why I'm not getting DNS resolution.

And, finally, when I posted the original error I was following Dave Sill's Life With QMail. It's a pretty good guide for those whose skills are current & proficient. I think we've established that mine are not. I used to have a copy of his book, "The QMail Handbook", which is much more complete & in depth. An excellent guide, howto & reference all in one. Well I haven't been able to find it so I ordered another copy which I should receive in the next day or so. In the mean time I was able to obtain a copy of John Levine's O'Reilly book, "QMail". I had made so many changes to the system & Levin's approach is not at all like Sill's, so I scrubbed my drive & started over with a clean install of Ubuntu 10.04.01, using Levine's book as a reference. Same results as far as the original error(s) are concerned. I still wasn't able to get email out to the world. You already know how we got from there to here, so I won't waste any time on that. What I wanted you to know is that the system changed, but the problem didn't. I'm certain there's a problem with qmail getting a DNS request either sent or answered but I don't know why.

And FINALLY <really really > If you don't have a copy of Levine's book, don't get one. I'm certain Dr Levine knows just about all there is to know about qmail, but his ability to convey that information leaves a bit to be desired. His approach to building the system is not intuitive. The process doesn't flow in a logical manner. At least not to my mind. There are also a lot of mistakes. /var/qmail/supervise/qmail/pop3d is NOT the same as /var/qmail/supervise/qmail-pop3d. And he often assumes the reader is on the same wavelength with him. That is definitely NOT the case. It may seem like I'm nitpicking, but this is a technical book, written by a man with a PhD. Also, it's several years old & in electronic form so you'd think most, if not all, mistakes would have been caught & corrected long ago. Sill's book, by contrast, is well written, flows from one step to the next in a logical manner, assumes nothing of the reader AND I don't recall more than 2 or 3 typos throughout. My first email server (long ago) was qmail built according to the proceedure laid out by Dave Sill in The QMail Handbook. It went smooth & quick & worked like a champ the 1st time through. No matter how this works out or what shape my system is in, when that book arrives I'm gonna scratch this one again use Sill's approach.

In the mean time, I'd REALLY like to know why I'm not getting DNS resolution.

Thanks for reading this (assuming you read it all). And let me say again, I'm sorry for not answering more completely before now.

Ed

 

[Noway2]

@400000004c9943332bd6e0a4 delivery 7: deferral: CNAME_lookup_failed_temporarily._(#4.4.3)

Something we can work with to go down a new path I think. First let me recap, just to clarify. Unfortunately, if you are operating a mail server on a host that has an IP from a public IP, most recipients (I should have said recipient rather than ISP) will not accept your mail. The reason is that >> 99.9% of the dynamic IP block are users and the only time their PCs attempt to send mail is when they are infected with a spam spewing virus host. Consequently, you get lumped in with that group by default. By contrast, static IP addresses, which you do pay more for, are considered more "business" grade. It sucks and it isn't fair, but it is.

The connection refused sounds like a closed port somewhere and this may still be a source of problems, but we can set it aside for right now.

By, the way, yes, I did read your entire post, you took the time to write it so I took the time to read it. It helps me to get an idea of what you are dealing with. Now, on to the error message.

I did a quick search on the error and it appears that it is a common hang up. Whats worse is that it appears to be one that can suddenly break on existing systems. The common culprit is that DNS records have, on average, grown in size and qmail can only handle 512 bytes at a time. If the record is bigger than that, which almost all of them are, the query fails. What happens is that it can't find the mx records in the response. Hence, by bypassing the DNS process, you work around that problem.

It looks like there are two solutions to this problem. One, you can patch qmail to fix the limitation. Two, the easier approach is to use a program called DJB's dnscache which feeds smaller pieces of information to qmail from your DNS. Apparently you want to try to use the FORWARDONLY mode, but this relies on having a good cache. Here is one link (

http://osdir.com/ml/mail.qmail.general/2002-07/msg01190.html)

I see that this is referenced in Life With Qmail, so perhaps we are barking up the wrong tree, but it seems to be the next thing to check.

 

[Noway2]

EDIT (a p.s.):

since you are using Linux, you can do an nslookup and then "set type=mx" without the quotes then enter a site like yahoo.com. This will show you the mx records. For yahoo, they are different than I was referencing in my post which I obtained from my own mail log of sending a message to my yahoo account.

Also, thought I would mention, yes, the mx record is the record that says "this is the mail server" with respect to the recipient. On YOUR end, you can have what is called an SPF record which allows the recipient to verify that your server has been declared as being a mail server. Spam zombies won't have this, so it is used as a check against this problem.

By the way, Bind9 is really easy to setup in Ubuntu. The first day I had a server up, I had dynamic DNS and DHCP linked in about an hour. It won't solve your current problem, but it is a fun thing to do. Here is a link I liked:

http://lani78.wordpress.com/2008/08/12/dhcp-server-update-dns-records/

and

http://lani78.wordpress.com/2008/08/09/setting-up-a-dns-for-the-local-network/

 

[cejohnsonsr]

Thanks for the info. I had just about come to some of the same conclusions. I spent quite a while playing with DNS to get myself up to speed again. Sort of. What I think may be happening (& I'd have mentioned it sooner if I had thought of it sooner) is something like this:

I have a public IP, but I'm not ready to pay for a "real" domain yet. So I've been using DynDns until I get it all going the way I want. In DynDns I have 2 hosts defined.

main_name.ath.cx has an A record
mail_name.ath.cx also has an A record
main_name.ath.cx also has an MX record that points to
mail_name.ath.cx

What I need is for my mail to be addressed to user@main_name.ath.cx (main_name.ath.cx being my domain & main_name being a subdomain in this context). When I dig main_name.ath.cx it shows an A record but no CNAME. which I think is what qmail wants for the domain. I have wilcard matching enabled, so when I dig mail.main_name.ath.cx MX I get mail_name.ath.cx as the MX for main_name.ath.cx and main_name.ath.cx as a CNAME for mail.main_name.ath.cx.

That ought to be about as clear as mud by now!

What I think has been happening is that the DNS lookups have been trying (& failing) to resolve an MX for ath.cx (which I don't own or control). I think what I have to do is tell qmail my FQDN is mail.main_name.ath.cx so it will return a CNAME record & use main_name.ath.cx as my domain instead of ath.cx. Either way it returns mail_name.ath.cx as the MX.

I'm pretty sure I had this same issue (or something similar) years ago. It's just been so long ago that I can't remember everything clearly.

I also got my new copy of The qmail Handbook today, so I have a much better guide. Haven't seen anything specifically about this so far, but it does mention having to use static SMTP routes if you don't get name resolution & I have seen several things that should make things go more smoothly this time. (Oh yeah. I scrubbed the system & started over fresh again so I won't have to worry about something failing because of something else I did but forgot.)

I haven't started building qmail yet. I wanted to wait until I had DNS whipped & then see what you thought before I proceeded.

So..... What do you think? Maybe I'm finally on the right track? The no CNAME error would fit & that would cause the original SMTP connection failure. All because it's trying to resolve a name server for ath.cx vs main_name.ath.cx.

Ed

 

[cejohnsonsr]

No. That didn't work either. The config script that does the DNS lookup apparently gets my IP# & does a reverse lookup so it returns MY ISP's name for the router. This is killing me.

Ed

 

[Noway2]

This is getting to be real twisted, so lets back up. In email, DNS is required for both the sender and recipient. For the sender, i.e. your outgoing, you need to be able to resolve other servers MX records. For receiving, others need to be able to resolve your mx record.

Did you say that the receiving portion was working correctly and that your problem is on the sending side?

Qmail is apparently funky about how it handles the reads of others MX records and this was the purpose of using the dns proxy, you reduce the amount of information qmail gets when looking up outgoing mail recipients. This shouldn't be impacted by your MX records, cnames, etc. A CNAME, by the way is just an alternate name (a pointer) to an A (address) record and shouldn't really be required as long as your A records are valid.

For the incoming mail, I think your setup where your MX record points to mail.main_name.ath.cx is fine and you can use a CNAME for mail.main_name.ath.cx to point it to main_name.ath.cx.

For one of my domains I use DynDNS also. I don't think I specified an MX record though and instead mail will just go to the root domain. This is fine too, especially for starters and is simpler.

 

[cejohnsonsr]

Before I say anything else, I want to say thank you for sticking with me on this. I appreciate your help.

You're right. This has gotten pretty convoluted. I've attempted to untangle things a bit & made a few discoveries in the process. I'll try to relate what I've done & what I've discovered since we last spoke.

For starters I completely reinstalled the OS again so I could be sure everything was fresh & nothing to interfere.

Once I read your last response I also changed my DynDNS by removing the MX record & the host it pointed to. It was the same as the main host anyway. And I disabled the wildcard option. I haven't done any testing since I made the changes.

Then I installed netqmail-1.06 per The qmail Handbook with a few minor changes due to version changes since its publication. Those changes came from Life With qmail which is more current. The changes are very minor.

When I had all of the processes up & running exactly as outlined in both QHB & LWQ I began testing per test.deliver provided with netqmail-1.06. Everything up to & including step 4 (local-error) went fine. Step 5 (local-remote) is when I started having trouble. Logs showed the same original error:

@400000004c9f8dc51ed82034 delivery 2: deferral: Sorry,_I_wasn't_able_to_establish_an_SMTP_connection._(#4.4.1)/

Here's where I think I've made a little progress:

When I installed the OS I gave it the hostname cuttinej. When I run /var/qmail/bin/config it can't find a DNS record because it's not a fully qualified domain name. So I ran /var/qmail/bin/config-fast cuttinej.ath.cx. <-- That is my FQDN in DynDNS. It occured to me that I should try cnfig with the FDQN so I changed /etc/hostname to cuttinej.ath.cx. After running config again I got the following:

root@cuttinej:/usr/local/src/netqmail-1.06# ./config
Your hostname is cuttinej.ath.cx.
Your host's fully qualified name in DNS is 12-34-56-78.abc.ispdomain.com.
Putting 12-34-56-78.abc.ispdomain.com into control/me...
Putting abc.ispdomain.com into control/defaultdomain...
Putting ispdomain.com into control/plusdomain...

Checking local IP addresses:
0.0.0.0: PTR lookup failed. I assume this address has no DNS name.
127.0.0.1: Adding localhost to control/locals...
192.168.1.10: PTR lookup failed. I assume this address has no DNS name.
12.34.56.78: Adding 12-34-56-78.abc.ispdomain.com to control/locals...

If there are any other domain names that point to you,
you will have to add them to /var/qmail/control/locals.
You don't have to worry about aliases, i.e., domains with CNAME records.

Copying /var/qmail/control/locals to /var/qmail/control/rcpthosts...
Now qmail will refuse to accept SMTP messages except to those hosts.
Make sure to change rcpthosts if you add hosts to locals or virtualdomains!

And therein lies the problem. qmail's DNS lookup returns the DNS name of my router as assigned by my ISP instead of my DynDNS hostname. I tried making using my IP in /etc/hostname & got the same result. (NOTE: I obviously changed the IP & FQDN in the uotput shown above. I'm not going to publish that for any reason. But the actual number & name were/are correct.)

So now I'm stuck again. How do I get qmail to resolve my DynDNS hostname instead of the router's ISP hostname? I tried pointing to DynDNS's name servers instead of my ISP's. That didn't work. I have port 53 forwarded from the router to my server even though I don't have anything listening on that port right now.

I know this will work. But I'm at a loss as to why it's not working.

Ed

 

[Noway2]

I am having a little confusion that you may be able to help clarify. When you say "DNS lookup returns the DNS name of my router as assigned by my ISP instead of my DynDNS hostname" is qmail doing a DNS lookup of your public IP address?

If so, yes, this is a reverse lookup, and unless this gets changed by the owner of the IP, in this case your ISP, you won't be able to get this to resolve to your FQDN through a public DNS. There are two ways around this that I can think of: 1 - put this in your host file, 2 - host your own authoritative DNS, such as BIND and have it resolve your IP.

I also almost hate to suggest it, but have you considered either Postfix, Zimbra, or Citadel? By the sounds of it, they may be easier to configure

P.S. - As an aside, re: "I'm not going to publish that for any reason. But the actual number & name were/are correct." I understand your position and typically don't post mine either. However, there is no real security benefit in not as the scan bots have found you a long time ago. The only thing that may happen if you post this information is that you will see a temporary spike in the scan rate.

 

[cejohnsonsr]

> is qmail doing a DNS lookup of your public IP address?

Again, that was part of MY question.

> There are two ways around this that I can think of: 1 - put this in your host file, 2 - host your own authoritative DNS, such as BIND and have it resolve your IP.

If you're referring to /etc/host, qmail doesn't consult this file to determine who I am. I don't really want to run a DNS server. That is the purpose of DynDNS. To resolve my IP to a name. It's not making the trip for email & I'm not sure why (http works fine), but it must be in the way qmail tries to perform the lookup.

> have you considered either Postfix, Zimbra, or Citadel? By the sounds of it, they may be easier to configure

Actually, I have considered postfix. (It's not any easier to configure. In fact I find it quite a bit more complex. One of the things I used to like about qmail was it's simplicity.) I decided to scrap qmail a couple of days ago. There is apparently no community support. A message to the qmail list has generated 0 response & you're the only one to answer this (or any other) forum post concerning qmail. I guess qmail is not what it once was. There doesn't appear to have been any new development or publication in a few years.

I've been working with postfix using falko's guide on HowToForge. Everything went according to plan (with a few minor hiccups) until testing. Test 1 fails. Can't get a response to ehlo via telnet. I just posted there. Hope I can get some help. One of the minor issues had to do with amavisd. Kind of a coincidence that the answer I found contained an entry from you.

Anyway, I guess we can put this one to bed. qmail isn't going to be what I was hoping/looking for.

Thanks for trying,

Ed

 

[Noway2]

Sorry to hear that qmail didn't work out for you. I think postfix will be a little easier to support at any rate. I have a fair bit of experience with it and I think a couple of others here do too. At a minimum, we should be able to help by comparing configuration files.