Interface won't accept (remote) connections

[Noway2]

I have what is turning into a real stumper of a problem for me and I am hoping someone else will see what I am missing.

I have a cable modem connection with a block of static IP addresses. I have two PCs, each with two NIC cards. On each PC, I have connected one of the NICs to the modem and one to a local LAN. In other words, each PC has one interface with a public IP address and one with a LAN IP address.

The addresses are public facing and have publicly accessible servers on them, so I won't bother to hide the IP addresses and will try to simplify the description.

PC has eth0 configured to the private LAN address of 192.168.0.50. It has eth1 configured to 96.10.215.210. I can access it remotely via the public IP or via the LAN on its LAN IP.

PC1 has a more complicated setup. It has eth0 and eth1, where eth0 has been converted into a brige (br0). It also has a virtual TAP device that is used for VPN that bridges to eth0. The eth0 / br0 is at LAN address 192.168.0.49. It uses a LAN router/gateway that is tied to the public IP of 96.10.215.214 and 192.168.0.254 on the LAN. I can access the machine via the public IP and port forwarding in the router. Eth1 is configured to the public IP of 96.10.215.211. The connection is from the NIC to the cable modem. I can't get it to respond from a remote address. There is no firewall, software or hardware in the way. I responds just fine from the LAN, which undoubtedly bridges at the modem. I have currently bound a (slave) DNS server to this interface and netstat is showing that it is listening at this address and on port 53.

If I do a traceroute to a working address, and the non working one, I get the same results all the way till the last hop at my IP, where it doesn't respond. It also won't respond to ping. If I ping it from within the LAN, it responds just fine. Traceroute from within the LAN shows only one hop as it will resolve to the same network.

The interface definition is provided below:

auto lo br0 tap0 eth1
#auto lo br0
iface lo inet loopback

iface br0 inet static
        address 192.168.0.49
        netmask 255.255.255.0
        network 192.168.0.0
        broadcast 192.168.0.255
        gateway 192.168.0.254
        bridge_ports eth0 tap0
        dns-nameservers 192.168.0.49
        dns-search debian.lan
        pre-up iptables-restore < /etc/iptables.rules

iface tap0 inet static
        address 172.17.0.1
        netmask 255.255.255.0
        pre-up openvpn --dev tap0 --mktun
        dns-nameservers 172.17.0.1
        dns-search debian.vpn

iface eth0 inet manual
        up ifconfig $IFACE 0.0.0.0 up
        up ip link set $IFACE promisc on
        down ip link set $IFACE promisc off
        down ifconfig $IFACE down

iface eth1 inet static
        address 96.10.215.211
        netmask 255.255.255.248
        network 96.10.215.208
        broadcast 96.10.215.215

here is a copy of the routing table

Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
96.10.215.208   *               255.255.255.248 U     0      0        0 eth1
192.168.0.0     *               255.255.255.0   U     0      0        0 br0
172.17.0.0      *               255.255.255.0   U     0      0        0 tap0
default         192.168.0.254   0.0.0.0         UG    100    0        0 br0

And here is a line from netstat that shows that the interface is listening:
tcp 0 0 96.10.215.211:53 0.0.0.0:* LISTEN

If I try to do an nslookup to this address, it times out saying that it can't connect to a server. If I use the .214 address that port forwards through the router to the other interface (which is also listening) it works just fine.

Does anyone have a clue as to why this interface doesn't work remotely, but does from wihtin the LAN?

 

[PeterHurst]

Hi Noway - forgive me I don't quite follow the reasoning and I'm not an expert on TAP and Bridge interfaces but I think your issue is more basic.

On the Linux box your gateway is 192.168.0.254 which doesn't exist as far as I can see.

For the box to speak to the internet the gw should be the internet gateway which you don't specify but should be a 96.x address and must be present on PC1.

It works across the router because the other PC's are on the same IP network and so will not require the gateway.

 

[Noway2]

Thank you for your reply. You noticed correctly, there is no gateway defined for that interface. You raised a point that only deepened the puzzle for me.

When I first set things up, I found that if I defined a gateway for the interface that it made the machine appear to stop communicating.

In light of your post, I performed an experiment on PC1, which has a simpler configuration (just eth0 and eth1), with the one interface defining the gateway in /etc/network.interfaces - the public IP address. This is a similar configuration to the one that I have been trying to fix where only one interface defines a gaetway. It seems reasonable that since they are defined to different network segments that each should have a gateway for that segment.

When I defined the gateway on the second interfaces with the gateway command (gateway 192.168.0.254), the machine stopped responding to eth0 on the public interface (with a gateway of 96.10.215.209 - the modem or ISP's router).

I discovered, though, that the machine still responded to eth1 on the 192 (LAN) range and I was able to SSH into it from another machine. I did this and dumped the routing table and got this:

localnet        *               255.255.255.248 U     0      0        0 eth1
192.168.0.0     *               255.255.255.0   U     0      0        0 eth0
default         192.168.0.254   0.0.0.0         UG    100    0        0 eth0
default         96.10.215.209   0.0.0.0         UG    100    0        0 eth1

As you can see defining the gateway for the other interface created TWO default gateways, both with a 0.0.0.0 mask, which says conflict to me. I manually removed the default gateway on eth0 and the machine immediately begame responsive on its public IP again.

I think you have found where my problem is and for that I thank you! The question now becomes how do I configure the interface to have a second gateway, that is network dependent? For example, it shouldn't be a default for all IP, but it should be preferential and have an appropriate net mask. If I just define 'gateway w.x.y.z' for both interfaces (where w.x.y.z) is the gateway for that interface I wind up with two defaults and it crashes.

 

[Noway2]

To provide closure to this issue, here is how I finally managed to resolve the problem.

It turns out that a machine can only have one (default) gateway defined and this is what it will use for all of its outgoing connections. What I didn't "get" at first is that this appears to also be the only line that it will listen to for Internet connections. Even when I had a service bound to and listening on a port at this address, connections would not establish. I suspect that what was happing is that incoming traffic was successful, but without the gateway, which is at the bottom of the public IP range, it didn't have a return path.

After doing some additional reading on connecting a single PC to multiple networks and with the experience I had trying to figure this out, I came up with the following: Each of the two servers have a public IP address assigned to one of their NIC cards. These interfaces are designated as static and a gateway is assigned to them. On the 'PC-2' which acts as the DNS and DHCP for the local LAN, I removed the gateway statement from the interfaces and moved it to the other NIC card. In the DHCP configuration, the 'option routers 192.168.0.254' is used and this assigned the WRT rotuer, which has a 3rd public IP, as the gateway for the LAN.

I used an external shell account on a different system to verify that I can now do DNS lookups on the slave DNS. One thing to note is that since recursive lookups are not enabled, it will only provide query information for that domain. This is a good thing as it should prevent anyone from trying to use it as their public DNS.

 

[theravager]

You can just add some static routes to the machines which will get around the single gateway issue

 

[Noway2]

Thank you for the suggestion. It turns out that this problem seems to have a desire to keep rearing its head. I recently discovered that one machine could not talk to the other, but that this was a problem only in one direction.

I ended up making the "public" interfaces for each service static IPs and made the rest of the LAN a separate one. This seems to work. The servers are now publicly accessible talk to each other as needed. As far as I can tell, it looks like the server will only 'listen' for remote connections on its primary or default gateway interface.