Setting up Netbackup to use/team multiple NICS to increase throughput

[systemengineertechie]

Scenario:

NBU 5.1 MP1

1 Master Server, Solaris 8.
2 Media Server, Solaris 8.

Need to find a way to team or use multiple nics on the Solaris media servers. Currently have (4) Gigabit nics on each media server. Using one on a primary network (seperate public network). And three on a backup network, same subnet. We are currenly using only one of the gigabit cards on the backup network, but would like to utilize the other two. We are usning DNS for both forward and reverse lookup. I have listed all DNS names for each backup NIC in the media servers servers list and the master server. When I try to do a backup over one of the other NICs that we currenly are not using I get a connection refused. When I run a bpcd log I see that it is having difficulty with the peername hostname and the DNS of the other gigabit card that I am trying to use. I would like to point different clients to different NIC's, by using these other DNS names on the client. Can netbackup handle this type of a setup?

I understand that I have the following options, but I would like to get it set up as mentioned above.

1. DNS round robin, but not sure the peername hostname issue would arise.
2. Teaming of the NICS at the switch
3. Teaming of hte NICS at the OS

Has anyone set this up?

 

[vm146971]

I have a private backup network and each media server is connected to it and the Public network as well. To force the clients to use the backup network and not the public network I edited the host file on the clients and associated the master/media server with IP of the NIC connected to the backup network. On the master/media server I configured the clients to be known as "host_nameB" and have DNS entries for each clients backup network NIC to be named "host_nameB". The B for backup. If you want to have 4 clients that you want to force to backup to for different NICS, and assuming each NIC has a unique IP you could configure the host file on each client to have an entry for the media server that is associated with the IP of the NIC you want to use.

 

[systemengineertechie]

I have tried associating the other NIC ip's in the clients host file, with no success. We currently have the media on both a public and a backup network and the clients to backup to both. I believe what happens when I attempt to use another NIC is that the OS and Netbackup use the first network card on each network and when I try to use another nic the client communication process checks the current OS hostname and compares it to what I am trying to use and I get a connection refused. Again I am not sure that netbackup can use two or more ip addresses on the same subnet for the client communication process. I have set up a bpcd log and when run a backup and it fails, the log shows that netbackup is trying to compare the hostname to the ip that I am trying to use and it refuses it, even that these are in DNS and in the server list.

 

[vm146971]

We have the same setup. Clients w/ large amounts of data are connected to the backup network, clients with small amounts of data are only on the public network, the media servers are connected to both networks. There is a setting

"Required Network Interface"

It might help. Look into trying this if you are not already using it.

There is a technote that refers to this feature:

http://support.veritas.com/docs/273433

 

[systemengineertechie]

We currently are using the REQUIRED_INTERFACE setting on some clients, but again we are looking to implement up to (3) Gigabit cards on the backup network and the REQUIRED_INTERFACE would restrict the media server to just one.

 

[vm146971]

Below is a similar configuration to the one that I use.
The Master server is also a media server.
The backup network is non routable.


master1
10.10.1.1 pub
10.10.2.1 backup
10.10.2.2 backup
10.10.2.3 backup
hostfile:
client1B 10.10.2.5
client2B 10.10.2.6
client3B 10.10.2.7


client1
10.10.1.2 pub
10.10.2.5 backup
host file:
Master1 10.10.2.1

client2
10.10.1.6 pub
client2B
10.10.2.6 backup
Host file:
Master1 10.10.2.2

client3
10.10.1.6 pub
client3B
10.10.2.7 backup
Host file:
Master1 10.10.2.3


In NBU you configure the clients as client1B, client2B and client3B, and add each client to a policy. That should work. NBU only knows the client by its backup network ip and name.
I do not use REQUIRED NETWORK-INTERFACE anywhere.

 

[systemengineertechie]

What OS is your Master/Media server? My configuration is about the same, but we use DNS and Solaris for the Master and Media servers. Again we have:

Master
10.x.x.x pub
101.x.x.x backup

Media Server
10.x.x.x pub
101.x.x.1 backup1
101.x.x.2 backup2
101.x.x.3 backup3

Client1
10.x.x.x pub
Client1B
101.x.x.x backup

With everything defined in DNS. But when I try to use anything other then 101.x.x.1 on the media server it won't work. Again I have defined all three of the backup nics in the master and media server lists. When I try to point to that backup nic on the client under server properties, I get a connection refused. I am wondering if Solaris will only present the port on the first nic on that subnet?

 

[vm146971]

I am running all w2k master and media servers. I have Netware clients that are multihomed and use the backup network. I will double check all my name resolution related configs in case I have missed something really obvious.

Maybe it is not a name resolution issue.
In your case the client is refusing the connection?
Is it refusing the connection to the Master or the Media server or both?

In the NBU client configuration:
Is the media server in the list of servers allowed to access the client?
does the client know itself as Client1B?
Can you use the 101.x.x.x network to connect to the client from the master server?

 

[systemengineertechie]

I just did another test on a client. On the client I removed the mediaserver1_b from the server list and added mediaserver2_b. I then ran a backup job. Here is the result of the BPCD log file.

08:59:30.770 [1080.3204] <2> bpcd main: offset to GMT 18000
08:59:30.770 [1080.3204] <2> bpcd main: Got socket for input 280
08:59:30.770 [1080.3204] <2> logconnections: BPCD ACCEPT FROM 101.x.x.1.995 TO 101.x.x.210.13782
08:59:30.770 [1080.3204] <2> bpcd main: setup_sockopts complete
08:59:30.785 [1080.3204] <2> bpcd peer_hostname: Connection from host mediaserver1_b.tecc.com (101.x.1x.1) port 995
08:59:31.801 [6904.4376] <2> bpcd valid_server: comparing masterserver_b and mediaserver1_b.tecc.com
08:59:31.801 [6904.4376] <2> bpcd valid_server: comparing mediaserver2_b and mediaserver1_b.tecc.com
08:59:30.785 [1080.3204] <2> bpcd main: output socket port number = 991
08:59:30.785 [1080.3204] <2> bpcd peer_hostname: Connection from host mediaserver1_b.tecc.com (101.x.x.1) port 995
08:59:30.785 [1080.3204] <2> bpcd main: Peer hostname is mediaserver1_b.tecc.com
08:59:30.785 [1080.3204] <2> bpcd main: Got socket for output 1924, lport = 573
08:59:30.785 [1080.3204] <2> bpcd main: Connected on output socket
08:59:30.785 [1080.3204] <2> bpcd main: Duplicated socket on stderr
08:59:30.785 [1080.3204] <2> bpcd main: <---- NetBackup 5.1 0 ------------initiated
08:59:30.785 [1080.3204] <2> bpcd exit_bpcd: exit status 46 ----------->exiting
08:59:30.785 [1080.3204] <4> bpcd exit_bpcd: FTL - BPCD EXIT STATUS 46

08:59:30.785 [1080.3204] <16> bpcd main: Server access denied

It appears to me that no matter what mediaserver I list as the server it defaults back to mediaserver1_b, even through it is not listed in the server list on the client. I tried a number of different configurations and it always defaulted back to the original mediaserver1_b NIC card?

 

[vm146971]

I had this issue too. We mistakenly had a _B appended to the Master/Media server name for the IP on the backup network.
The clients refused the connection every time.

They knew about MasterServer and MediaServer. They did not know about MasterServer_B or MediaServer_B, and NBU did not know about them either. The clients and would not let them connect.

If this is what you are doing, and I hope it is,it definitly will not work, but the fix is simple.

If the problem is that you are defining the Media Servers IP on the backup network as "MediaServer_B", try defining the Master and Media servers IP addresses on the backup network as they are known on the public network:

"MediaServer" without the _B.

That is why we ended up using host files on the clients to define the media servers connection to the backup network.


Media server1 is multihomed:
nic1 is on 101.x.x.x the public network.
there is a DNS entry for media server1 on 101.x.x.x.
All servers connect to MediaServer1 via the DNS entry.

The exeption to this rule is:

Except the servers that use the backup network. They connect to MediaServer1 via the Host file entry:
102.x.x.x MediaServer1 not MediaServer1_B

If you have 4 clients that you want to connect to 4 different NICs you would have a host file oin each that defined MediaServer1 by the IP of the NIC you want to connect to.

client1
102.x.x.1 MediaServer1

client2
102.x.x.2 MediaServer1

client3
102.x.x.3 MediaServer1

client4
102.x.x.4 MediaServer1

In all cases the clients connect to MediaServer1 there is no entity known as MediaServer1_B. In all cases the Master and Media server names need to be the same for all clients regardless of the network they use to backup. The client names are the only names that should have the _B appended to name for theIP address on the backup network. You should be able to use DNS or a host file to resolve the names to clients.

 

[systemengineertechie]

That is a good call. I placed the media server 2nd backup nics in the clients host file. As mediaserver1 - 2nd backup IP address. However it still was trying to use the original IP address. It looks like what is happening is based on the storage unit. We have the storage units using the primary NIC DNS name. For some reason it uses the first NIC for the backup network.

If I use netstat -rn I see the following

101.x.x.1
101.x.x.2
101.x.x.3

And I believe it defaults to using the first one listed, which is 101.x.x.1. I can't seem to get it to use one of the other IP when BPCD starts. So on the client I put

101.x.x.2 mediaserver1_b2

Only because this is the host name that BPCD is always trying to use which is based on the storage unit hostname.

I also tried

101.x.x.2 mediaserver1

Again it was looking for the mediaserver1_b1 IP address.

 

[vm146971]

did you flush the dns cache on the servers after you made the change?

 

[systemengineertechie]

I flushed the DNS cache and I still have the same issue. It looks like to me that BPCD gathers the peername from the storage unit hostname through DNS and compares it to what you have in the server list. If the correct name is in the server list then it will proceed with the backup. Again I am not sure why it chooses the first nic listed in the OS. (see earlier comments)

 

[vm146971]

Have you thought about teaming the NICS and the ports they attach to on the switch? Compaq refers to this as Switch Assisted Load Balancing.

Heres a white paper:
ftp://ftp.compaq.com/pub/products/servers/networking/TeamingWP.pdf Page 21 is similar to my set up.

This is actually how my environment is set up. I have the NICS and the ports they attach to Teamed. I have one IP adrdress for the Media server on the backup subnet.

Since all your servers are on the same subnet, you could have all 3 NICS on the media server assigned the same IP. Then you would not have to worry about cofiguring different clients to attach to a particular NIC.

Is this an option in your environment?

 

[vm146971]

Maybe the default in Solaris is to use the 1st NIC address it finds to send data out to a subnet unless you tell it to use a different NIC

In windows you can force traffic to go out a particular NIC based on IP addresses by creating a persistant route on the server:

Example:

Define a route on the media server for each NIC on the backup network that you want to use to connect to the client. I do not know how you do this in Solaris, but it would look something like this in Windows:

Destination: 101.x.x.5 Client1
gateway 101.x.x.1 NIC 1 on Media server
interface 101.x.x.1

Destination: 101.x.x.6 Client2
gateway 101.x.x.2 NIC 2 on Media server
interface 101.x.x.2

Destination: 101.x.x.7 Client3
gateway 101.x.x.3 NIC 3 on Media server
interface 101.x.x.3