Inconsistent results from expect_out(buffer)

[zombieZero]

I've encountered an interesting problem...I have a generic proc that sends commands to a cli, and returns the responses. Among other things, I use this to pull the current configuration from the box. In theory, this proc should work no matter what device I use it on, and for the most part this is the case, but recently I've turned it on a new CLI and I'm hitting a strange problem.

The command goes down, and it does generate a response; that part does still work, but the response which returns is inconsistent. The re I use is:

expect {
-i $sid -re "(\[^\r]*)\r\n" {
lappend menuList $expect_out(buffer)
}

...according to 'Exploring Expect', this should mean 'everything'...so basically, just capture the whole thing in the buffer and return it to me. About half of the time it does, but the other half of the time I get either half the output, or none of it.

The longer the configuration is, the worse the results. I thought at first this was a buffer size problem, so I increased the max_match size to 4000 (from the default of I believe 2000), but this did no good.

Does anyone have an angle on why this might be happening? Are there any known control characters or other strange output I might be getting back that could explain the inconsistent results of the expect_out(buffer)?

 

[zombieZero]

Some more info...first, the configuration output is sometimes long enough that the user is prompted to hit the space bar in order to continue. This is addressed with the following line:

-i $sid -re "\<space\>" {
send -i $sid " \r"
}

This comes just before the 'match everything' line.

I set exp_internal to try and get a better idea of what was happening in there and I suspect it may be related to this (sorry if this is too much stuff to look through, I wanted to be thorough);

expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) " = next page; <enter> = next line; <Q> = stop) --\u001b[m\r\u001b[J ip address 60.1.1.254

255.255.255.0\r\r\n router-discovery maximum-interval 1200\r\r\n router-discovery minimum-interval 500\r\r\n

exit\r\n\rrouter rip\r\r\n default-metric 10\r\r\n ecmp\r\r\n enable\r\r\n split-horizon poison-reverse\r\r\n

network 100.1.1.254 0.0.0.255\r\r\n network 110.1.1.254 0.0.0.255\r\r\n network 50.1.1.254 0.0.0.255\r\r\n network

60.1.1.254 0.0.0.255\r\r\n exit\r\n\rrouter ospf 5\r\r\n distance 5\r\r\n dead-interval 10\r\r\n hello-interval

5\r\r\n inherit-metric\r\r\n network 60.0.0.0 0.255.255.255 area 0.0.0.0\r\r\n exit\r\n\rrouter bgp 1\r\r\n

preference2 0\r\r\n\u001b[7m-- MORE (<space>"
send: sending " \r" to { exp6 }

expect: does " = next page; <enter> = next line; <Q> = stop) --\u001b[m\r\u001b[J maximum-routes 1000\r\r\n\u001b[7m--

MORE (<space> = next page; <enter> = next line; <Q> = stop) --\u001b[m" (spawn_id exp6) match glob pattern "^\r \r"?

no
"^--MORE--"? no
"<space>"? yes <------*****MATCHES THIS BUT NOTHING ELSE
expect: set expect_out(0,string) "<space>"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) " = next page; <enter> = next line; <Q> = stop) --\u001b[m\r\u001b[J maximum-routes

1000\r\r\n\u001b[7m-- MORE (<space>"
send: sending " \r" to { exp6 }
expect: does " = next page; <enter> = next line; <Q> = stop) --\u001b[m" (spawn_id exp6) match glob pattern "^\r

\r"? no
"^--MORE--"? no
"<space>"? no
"([^\r]*)\r\n"? no

...for some reason the above portion doesn't match even for the 'everything' re.

If it helps, another consistent behavior is occasionally nothing at all gets captured (except the prompt) and the expect_out(buffer) gets set to a bunch of escape characters, carriage returns and nextlines:

expect: does "" (spawn_id exp6) match glob pattern "^\r \r"? no
"^--MORE--"? no
"<space>"? no
"([^\r]*)\r\n"? no
"^\rprompt[^#]*[#$]"? no
"^prompt[^>]*"? no

prompt#
prompt#
expect: does " \u001b[C\u001b[C\r\n\rprompt# \u001b[C\u001b[C\r\n\rprompt#" (spawn_id exp6) match glob pattern "^\r

\r"? no
"^--MORE--"? no
"<space>"? no
"([^\r]*)\r\n"? yes
expect: set expect_out(0,string) " \u001b[C\u001b[C\r\n"
expect: set expect_out(1,string) " \u001b[C\u001b[C"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) " \u001b[C\u001b[C\r\n"

expect: does "\rprompt# \u001b[C\u001b[C\r\n\rprompt#" (spawn_id exp6) match glob pattern "^\r \r"? no
"^--MORE--"? no
"<space>"? no
"([^\r]*)\r\n"? yes
expect: set expect_out(0,string) "prompt# \u001b[C\u001b[C\r\n"
expect: set expect_out(1,string) "prompt# \u001b[C\u001b[C"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "\rprompt# \u001b[C\u001b[C\r\n"

expect: does "\rprompt#" (spawn_id exp6) match glob pattern "^\r \r"? no
"^--MORE--"? no
"<space>"? no
"([^\r]*)\r\n"? no
"^\rprompt[^#]*[#$]"? yes
expect: set expect_out(0,string) "\rprompt#"
expect: set expect_out(spawn_id) "exp6"
expect: set expect_out(buffer) "\rprompt#"

 

[Bong]

I don't know anything about EXPECT so I don't know if I can be of help. On the other hand, you don't seem to be getting too many nibbles, so how about this:

It seems the your regular expression (assuming that's what "-re" means) returns anything in the buffer that has the form: anything but a "return", followed by anything, followed by a "return", followed by a "new line". Could it be that your latest CLI (client?) doesn't use CR/LF?

Bob Rashkin
rrashkin@csc.com

 

[zombieZero]

Thanks Bong...that actually covered part of the problem; this CLI sometimes uses /r/n, other times just /r, so I added an extra catch in there and that helps...still, it seems that whenever I get a hit on the 'space' match, the output was getting dumped so I added another line to that match saying check expect_out(buffer); if it has extraneous info in it trim it off then add it to the return list...the only problem now is that when this occurs it does get added, but the format of the entry gets changed from:

} {\rinterface ge.1.1\r\r
} { ip address 100.1.1.254 255.255.255.0\r\r
} { exit\r
}
} {\rinterface ge.1.2\r\r
} { ip address 110.1.1.254 255.255.255.0\r\r
} { exit\r
}

to:

} {\rinterface ge.1.1\r\r
} { ip address 100.1.1.254 255.255.255.0\r\r
} { exit\r
\rinterface ge.1.2\r\r
ip address 110.1.1.254 255.255.255.0\r\r
exit\r
-- MORE (<space>}

...So in the second instance (when the space is sent to make the output continue), the chunk in expect_out(buffer) gets added as part of the 'exit' line of the previous interface, which is causing me all kinds of heartburn.

 

[Bong]

I'm afraid that's going to require someone who knows from EXPECT.

Bob Rashkin
rrashkin@csc.com

 

[zombieZero]

Well, I don't know if I solved it, but I found a way around it anyway...I took the expect_out(buffer), split it at the carriage returns, then fed the individual elements back into the list...it required some cleanup of some extraneous stuff, but I got it to work.

If anyone has an angle on this, though, I'd appreciate it...I can't help but think there's a cleaner way to do it.

Thanks again for taking a stab at it Bong.