Expect script help

[alan147]

I have an expect script that uses sftp to download files from a remote windows host.

I gave just moved this to a new server RHEL 6.1 and cannot get the script to work. I can manually sftp to the remote host and list the directory tree.

The expect script produces the following error:

Permission denied (publickey,password).
Couldn't read packet: Connection reset by peer
send: spawn id exp4 not open
while executing
"send "\password\r""
(file "./xxxxxxx.sh" line 21)
with sftp -v in the expect script I get the following information:


OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug1: Connecting to 62.189.104.137 [62.189.104.137] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type -1
debug1: identity file /root/.ssh/id_dsa type 2
debug1: Remote protocol version 2.0, remote software version WRQReflectionforSecureIT_7.1.2 Build 835
debug1: no match: WRQReflectionforSecureIT_7.1.2 Build 835
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_5.3
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5 none
debug1: kex: client->server aes128-ctr hmac-md5 none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Host 'xxx.xxx.xxx.xxx' is known and matches the DSA host key.
debug1: Found key in /root/.ssh/known_hosts:2
debug1: ssh_dss_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug1: Offering public key: /root/.ssh/id_dsa
debug1: Authentications that can continue: publickey,password
debug1: No more authentication methods to try.
Permission denied (publickey,password).
Couldn't read packet: Connection reset by peer
send: spawn id exp4 not open
while executing
"send "\password\r""
(file "xxxxxxx.sh" line 21)


Any ideas?

Thanks

Alan

 

[Annihilannic]

Well, it doesn't look like an expect issue... it's just a problem with your SSH keys. Have you checked that the ~/.ssh/authorized_keys file on the remote system contains the right public key?

Have you tried running sftp manually?

Annihilannic
[small]tgmlify - code syntax highlighting for your tek-tips posts[/small]

 

[Annihilannic]

Incidentally there is a FAQ which may help you debug SSH authentication issues.

Annihilannic
[small]tgmlify - code syntax highlighting for your tek-tips posts[/small]

 

[alan147]

Manual sftp works fine.

Alan

 

[tarn]

Manual sftp works fine.

What without asking for password ?

If its prompting your for password then its likely to be the shared public key is missing from the new host ..... Either set up a new key or get the public key from the old system across.

Laurie.

 

[alan147]

Manual sftp requires a password. I need the expect script to use password authentication.

Alan

 

[alan147]

Manual sftp -vvv user@xxx.xxx.xxx.xxx

Produces this output

Connecting to xxx.xxx.xxx.xxx...
OpenSSH_5.3p1, OpenSSL 1.0.0-fips 29 Mar 2010
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Applying options for *
debug2: ssh_connect: needpriv 0
debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type -1
debug3: Not a RSA1 key file /root/.ssh/id_dsa.
debug2: key_type_from_name: unknown key type '-----BEGIN'

lines removed

debug3: check_host_in_hostfile: filename /root/.ssh/known_hosts
debug3: check_host_in_hostfile: match line 2
debug1: Host 'xxx.xxx.xxx.xxx' is known and matches the DSA host key.
debug1: Found key in /root/.ssh/known_hosts:2
debug2: bits set: 495/1024
debug1: ssh_dss_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: Wrote 16 bytes for a total of 997
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug3: Wrote 48 bytes for a total of 1045
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /root/.ssh/id_rsa ((nil))
debug2: key: /root/.ssh/id_dsa (0xb9147008)
debug3: Wrote 64 bytes for a total of 1109
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug3: no such identity: /root/.ssh/id_rsa
debug1: Offering public key: /root/.ssh/id_dsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug3: Wrote 528 bytes for a total of 1637
debug1: Authentications that can continue: publickey,password
debug2: we did not send a packet, disable method
debug3: authmethod_lookup password
debug3: remaining preferred: ,password
debug3: authmethod_is_enabled password
debug1: Next authentication method: password
user@xxx.xxx.xxx.xxx's password:

but when called via the expect script I get this output:

lines deleted


debug1: Host 'xxx.xxx.xxx.xxx' is known and matches the DSA host key.
debug1: Found key in /root/.ssh/known_hosts:2
debug2: bits set: 511/1024
debug1: ssh_dss_verify: signature correct
debug2: kex_derive_keys
debug2: set_newkeys: mode 1
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: Wrote 16 bytes for a total of 997
debug2: set_newkeys: mode 0
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug3: Wrote 48 bytes for a total of 1045
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug2: key: /root/.ssh/id_rsa ((nil))
debug2: key: /root/.ssh/id_dsa (0xb8bbe008)
debug3: Wrote 64 bytes for a total of 1109
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey
debug3: authmethod_lookup publickey
debug3: remaining preferred: ,gssapi-with-mic,publickey
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug3: no such identity: /root/.ssh/id_rsa
debug1: Offering public key: /root/.ssh/id_dsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug3: Wrote 528 bytes for a total of 1637
debug1: Authentications that can continue: publickey,password
debug2: we did not send a packet, disable method
debug1: No more authentication methods to try.
Permission denied (publickey,password).
Couldn't read packet: Connection reset by peer
send: spawn id exp4 not open
while executing
"send "\password\r""
(file "./script.sh" line 15)


The expect script is below
#!/usr/bin/expect -f

#runs sftp in backgroup using batch script as input called batch.BAT

set env(TERM) vt100

spawn /usr/bin/sftp -vvv -b /usr/local/bin/download/scripts/batch.BAT user@xxx.xxx.xxx.xxx

sleep 20

expect -exact "user@xxx.xxx.xxx.xxx's password: "

sleep 20

send "\password\r"

expect eof

 

[feherke]

Hi

[ul]
[li]Why are you using those [tt]sleep[/tt]s ?[/li]
[li]Is your password really \password ? If it includes the leading backslash ( \ ) too, escape it with another backslash.[/li]
[/ul]


Feherke.

 

[alan147]

The sleeps are to allow time for the remote server to respond. The password is not password, I didn't take the \ out when I sanitised the password.

Alan

 

[feherke]

Hi

The sleeps are to allow time for the remote server to respond.

Pointless. The [tt]expect[/tt] command will wait as necessary.

[tt]expect [[-opts] pat1 body1] ... [-opts] patn [bodyn]
waits until one of the patterns matches the output of a spawned
process, a specified time period has passed, or an end-of-file is
seen. If the final body is empty, it may be omitted.[/tt]

Feherke.

 

[alan147]

OK thanks, I've removed them.

Alan

 

[feherke]

Hi

Any change in the output now ? Specifically the "Couldn't read packet: Connection reset by peer" line is still there ?


Feherke.

 

[alan147]

No, it is still the same.

debug3: Wrote 64 bytes for a total of 1109
debug1: Authentications that can continue: publickey,password
debug3: start over, passed a different list publickey,password
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey
debug3: authmethod_lookup publickey
debug3: remaining preferred: ,gssapi-with-mic,publickey
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug3: no such identity: /root/.ssh/id_rsa
debug1: Offering public key: /root/.ssh/id_dsa
debug3: send_pubkey_test
debug2: we sent a publickey packet, wait for reply
debug3: Wrote 528 bytes for a total of 1637
debug1: Authentications that can continue: publickey,password
debug2: we did not send a packet, disable method
debug1: No more authentication methods to t
Permission denied (publickey,password).
Couldn't read packet: Connection reset by peer
send: spawn id exp4 not open
while executing
"send "\password\r""
(file "/usr/local/bin/download/bin/script.sh" line 15)

 

[tarn]

OK I've been google'n your error ( send: spawn id exp4 not open)

This would appear to be that the ssh session is not yet ready for the next command ..... It may be waiting for you to confirm that the public key it has does not match the host sending the expect script ...

So worth checking and setting this if its not already done in the servers sshd_config and restarting sshd

stricthostkeychecking=no

Can I assume you have Tested the expect script actually from the host that you are logging into (it sometimes helps to rule out host-key issues).

Other thoughts are:

Who's running the expect script and does expect know that users profile (just to avoid problems when any spawning sub shell issues)

When you sftp in manually are you doing it from the same "expect" host and as the user that is running the "expect" script?


It does point towards the "stricthostkeychecking=no" setting, you can/could check/remove the host key string from known_hosts file also just in case .....

Laurie.

 

[alan147]

I've set stricthostkeychecking=no to no effect and deleted the key from the known_hosts file. The expect script worked on another but I needed to move it to the current one. I am running the script and the manual sftp from the same server.

Alan

 

[tarn]

OK .... well thats all ruled out however;

if you google the error it would indicate that the ssh (sftp) session is not ready before you start passing it something to do so (I know nothing about expect) can you complete your login and then call your batch (BAT) script

If you google "send: spawn id exp4 not open" you will see loads of this issue from expect/Linux

various reasons but ssh not being ready before you pass the commands is one of them ...

Laurie.

 

[Annihilannic]

Strange one... what was the script running on before you moved it to this RHEL6.1 system? What versions of ssh and expect on the new and old systems?

Annihilannic
[small]tgmlify - code syntax highlighting for your tek-tips posts[/small]

 

[alan147]

The rpm packages on the new server are:

openssh-server-5.3p1-70.el6.i686
openssh-5.3p1-70.el6.i686
openssh-clients-5.3p1-70.el6.i686
expect-5.44.1.15-2.el6.i686

OS Oracle Linux Server release 6.2

Old server:

openssh-server-3.9p1-8.RHEL4.24
openssh-askpass-3.9p1-8.RHEL4.24
openssh-3.9p1-8.RHEL4.24
openssh-askpass-gnome-3.9p1-8.RHEL4.24
openssh-clients-3.9p1-8.RHEL4.24
expect-5.42.1-1

OS Enterprise Linux Enterprise Linux AS release 4 (October Update 6)

Alan