HACMP - SSH Key problem while using service name 2

[KPKIND]

Hi,

I am trying to connect to various HA clusters using the SERVICE NAME that they are running on these clusters using SSH and execute some commands.

Problem: When the service is running on the primary node, ssh known_hosts file gets updated with the key from the primary node of the cluster and when the service falls over to the other nodes in the cluster, I cannot connect to the service until I change the identification key in my known_hosts file.

Since the connection I am making is using a script, I am trying to find any other way to get around this problem.

telnet is disabled for security reasons. I thought of dsh as well but I believe dsh also internally uses ssh which is of no use.

Any ideas are appreciated.

TIA
Cheers
Kumar

 

[abubasim]

Create global ssh_known_hosts and ssh_known_hosts2 files in /etc/ssh and make sure the keys have all possible hostnames and IP addresses listed. The format of these files are the same as the personal known_hosts in your ~/.ssh directory.
Each key can have multiple names and addresses listed comma-separated:

hostname1,hostname2,ipaddr1,ipaddr2 ssh-rsa AAAlongkeycontinuinghere

 

[khalidaaa]

Oh yeah abubasim did that on our clusters and they are working like a charm

Thanks abubasim.

Regards,
Khalid

 

[ogniemi]

you should keep all types of host_keys (RSA1, RSA, DSA) THE SAME on all nodes of the cluster!!!

In the other ways, distribute host_keys files (from location specified in sshd_config):

ssh_host_dsa_key
ssh_host_dsa_key.pub
ssh_host_key
ssh_host_key.pub
ssh_host_rsa_key
ssh_host_rsa_key.pub

from one cluster node to the others. You will have to restart sshd on all nodes of the cluster to which above keys were copied, eg. running command on these nodes:

stopsrc -s sshd;startsrc -s sshd

Of course you should update or remove known_hosts files after that. But after it will be done you will never have problems with host keys after service IP is takeovered to secondary nodes.

I have also to agree with abubasim that global known_hosts which is called ssh_known_hosts - wich is generally located in the same dir sshd_confing resides - is very good idea I also use.

When you have synced host keys on all nodes of your cluster, go to the ssh config directory on node node, eg. /usr/local/etc/ and run following commands:

cat ssh_host_*.pub > ssh_known_hosts_tmp
sed s/^/"$(/usr/sbin/cluster/utilities/cllsif|grep -w ether|awk '{printf $1","$7","}'|sed s/\,$/\ /g)"/g ssh_known_hosts_tmp > ssh_known_hosts
chmod 444 ssh_known_hosts

You can of course add extra entries (coma separated) to begin of each line in the global ssh_known_hosts (eg. FQDN)

Next, propagate the prepared global ssh_known_hosts file to the other nodes in the cluster:

scp -p /usr/local/etc/ssh_known_hosts clnode2:/usr/local/etc/
scp -p /usr/local/etc/ssh_known_hosts clnode3:/usr/local/etc/
etc.

There is also StrictHostKeyChecking parameter in ssh_config (client config file - global is located in the same dir sshd_config eg. /usr/local/etc/).

When you set 'StrictHostKeyChecking no' then new host_keys will automatically be written to user's known_host file - no prompt for confirmation before adding.

Some people also enable 'HashKnownHosts yes' (available since openssh v.4) in ssh_config - it makes known_hosts files more secure.

 

[ogniemi]

forgot to say - to hash global known_hosts_file it is need to run following command:

ssh-keygen -H -f /usr/local/etc/ssh_known_hosts

after that, you can remove unhashed copy which was saved to /usr/local/etc/ssh_known_hosts.old

 

[abubasim]

@ogniemi: Good stuff, that.

I forgot that IPAT between nodes requires identical host keys.

niemi sounds Finnish.

 

[abubasim]

@khalidaaa,

You're welcome. BTW, do you still need to cut your Umm Ali with a knife?

(Umm Ali is a classic Arabic desert similar to bread pudding.)

 

[KPKIND]

Hi Guys,

Thanks you abubasim & ogniemi for your responses. Very much appreciated.

The option of copying the hostkey from one node of the cluster to all the other nodes in the cluster has been ruled by our security team although I have told them I have used this in the past. If there is a real problem I am sure most of the companies would not adopt this... if at all you know of any consequences of copying the same host key, please can you let me know.

I never had an idea of ssh_known_hosts option which I have used currently and it works like a trick.

That has really done the job.

Thanks a lot guys...

Cheers