Picking multiple matches out of a single string 1

[rubberscissors]

This seems like it should be simple but for the life of me I can't seem to peg this; I'm using expect to telnet into a device and issue commands. What I want to do is then expect certain things in the output, and then execute other commands based on what it sees. The problem is, I need it to look for multiple things in the same output (call them klatu, barada, and nikto). So I tried the following:

expect "prompt" { send "the command\r" }

expect { klatu { incr klatucheck +1
exp_continue }
barada { incr baradacheck +1
exp_continue }
nikto { incr niktocheck +1
exp_continue }

}

Since initially I just wanted to check for certain conditions, then react to them individually afterwards. This didn't work, because it still only went through the output once and many 'niktos' were passed by during the 'barada' screening...so, I just wrote the output to a file and screened the file multiple times, which works, but I couldn't help but think the file wasn't necessary...

So, I decided to try it this way:

set matchlist { klatu barada nikto }

set x $expect_out(buffer)

set i 1
foreach item $matchlist {

set match$i 0

if [ string match "*$item*" $x ] { set match$i 1 }

incr i +1

}
if { $match1 != 0 } { klatucheck $x }
if { $match2 != 0 } { baradacheck $x }
if { $match3 != 0 } { niktocheck $x }

Then, I passed the expect output buffer to the different 'check' proceedures as a string. So far so good.

The problem arises when I then screen the output buffer in the 'check' proceedures. For my purposes, I already know at least one I/O slot was in the 'klatu' state. Now I want to know which I/O slots are in the 'klatu' state. I run the 'klatucheck' proceedure, and do the following:

if [ string match "*klatu*" $x ] {
set index [ string first "klatu" $x ]
set a [ expr $index - 8 ]
set b [ expr $index - 7 ]
set i [ string index $x $a ]
set o [ string index $x $b ]
set io$count $i$o

}

This returns a two digit number as $io1, the I/O slot which is in the 'klatu' state...unfortunately, it only grabs the first instance of 'klatu' so any other slots in that state are missed completely. I need it to be able to keep screening through the string until all instances have been found, then I'll compile the slot numbers into a list, then check the slots on the list using other commands.

Does anyone out there know how I can check the whole string instead of just stopping after the first match? Or have I just started a journey down the wrong path? Any help would be appreciated!

 

[marsd]

You can use expects -re feature to catch multiple matches
more succinctly.
You can also nest your "barada" search within the match
for either of the other two.

Your explanation is not very helpful.
Could you give us a sample of some input and output
patterns you have trouble getting concise matches of?

 

[AviaTraining]

We've got several issues here to deal with. And in general, rubberscissors, I recommend that you read through the entire string reference page at length, as there are several string commands that make it much easier to manipulate strings than the way you're doing it. Also, read up a bit on arrays in Tcl, as they're a much easier and more natural way of dealing with collections than the generated variable names that you're using. (For example, "io($count)" vs. "io$count".)

First, let's address the issue of searching for multiple occurrences of a substring within a string. Unfortunately, string first isn't quite as flexible about this as I'd like. It returns the index of only the first match, or -1 if it doesn't find a match. So, taking advantage of the "-1" return value can eliminate your calls to string match in this particular application. Additionally, we can make use of the optional final argument to string first, which is the index within the string where string first starts looking for a match.

We'll also make use of the string range command, which returns a range of characters from an input string, rather than just a single character. This eliminates the need for your 2 string index calls where you then simply concatenate the characters.

Putting all this together we can write the following procedure which accepts a string and a search string as arguments, and returns a list of the starting indices of all occurrences of the substring (or an empty list if no occurrences are found):

proc findAll {str substr} {
  set indices [list]
  set loc 0
  while {[set loc [string first $substr $str $loc]] != -1} {
    lappend indices $loc
    incr loc
  }
  return $indices
}

Testing out this procedure, we get:

[tt]% findAll "This is a test, isn't it?" "is"
2 5 16
% findAll "This is a test, isn't it?" "blah"
% findAll "This is a test, isn't it?" "i"
2 5 16 22
[/tt]

Looks good. Then, we can take some data, search for our string, and then loop around the returned list of indices to grab the slot substrings. For example, given this data:

set input "Status: 00 slot klatu
Status: 01 slot waffle
Status: 02 slot nikto
Status: 03 slot klatu
Status: 04 slot waffle
Status: 05 slot barada
Status: 06 slot klatu
Status: 07 slot barada"

We can do this:

[tt]% findAll $input "klatu"
16 83 151
% [ignore]set slots

• [/ignore]

• 
  % [ignore]foreach index [findAll $input "klatu"] {[/ignore]
  > [ignore]lappend slots [string range $input \[/ignore]
  > [ignore][expr {$index - 8}] [expr {$index -7}]][/ignore]
  > [ignore]}[/ignore]
  % [ignore]puts $slots[/ignore]
  00 03 06[/tt]
  
  Okay, so much for the "simple" approach. Another powerful approach is to use regular expressions to parse your data. But regular expressions are a complex topic, too much so to get into in any depth here. However, I can go ahead and show you a couple of tricks with regular expressions that you might find useful here.
  
  The regexp command searches a string for a regular expression match. It takes various options, including an -all flag to find all matches, and an -inline flag to return the matches as a list instead of storing them in a variable. The following example shows a way to search for all the "klatu" slots:
  
  [tt]% regexp -all -inline {(..).{6}klatu} $input
  {00 slot klatu} 00 {03 slot klatu} 03 {06 slot klatu} 06[/tt]
  
  You could process this list with a foreach loop as follows:
  
  

  set slots [list]
  foreach {junk slot}   [regexp -all -inline {(..).{6}klatu} $input] {
      lappend slots $slot
  }

  
  Alternately:
  
  [tt]% regexp -all -inline {(..).{6}(klatu|barada|nikto)} $input
  {00 slot klatu} 00 klatu {02 slot nikto} 02 nikto {03 slot klatu} 03 klatu {05 slot barada} 05 barada {06 slot klatu} 06 klatu {07 slot barada} 07 barada[/tt]
  
  And:
  
  

  set slots(klatu) [list]
  set slots(barada) [list]
  set slots(nikto) [list]
  foreach {junk slot value}   [regexp -all -inline {(..).{6}(klatu|barada|nikto)} $input] {
      lappend slots($value) $slot
  }

  
  Finally, a comment about using multiple patterns with an expect command. Whenever the expect command receives a chunk of data from the spawned process, it immediately tests the data it's received so far against all the patterns in the order that the patterns are listed. If there are no matches, it grabs another chunk of data and tries again until it's either successful with a match or the timeout expires. Thus, we have the following example that you might find surprising:
  
  [tt]expect1.1> expect {
  +> bad { send "That's bad\n" }
  +> good { send "That's good\n" }
  +>}
  Sometimes it's good to be bad.
  That's bad[/tt]
  
  Even though "good" appeared in the data stream before "bad", in the chuck of data that expect read, it was able to match the "bad" pattern, and so didn't bother checking the "good" pattern.
  
  So, given this behavior, you might very well find it easier to continue grabbing the output in one big string, and then processing it afterwards like you're doing, rather than trying to do the parsing within your expect patterns like marsd suggested. - Ken Jones, President, ken@avia-training.com
  Avia Training and Consulting,
  http://www.avia-training.com
  866-TCL-HELP (866-825-4357) US Toll free
  415-643-8692 Voice
  415-643-8697 Fax

 

[marsd]

Once again post some program output if you want.
We can see if in-detail string searching is the answer.
There may be a more concise matching scheme that
alleviates the need for detailed buffer processing.

 

[rubberscissors]

Thanks, AviaTraining, as always for the time and effort...I'd been grinding through my string options for a while now. I couldn't find any other commands that seemed to suit my needs - I use a string match earlier in the script to determine whether the state exists at all, then I needed another to pick out the instances. string index, compare, length, range...none seemed to do what I'm looking for. Your snippet:

proc findAll {str substr} {
set indices

• 
  set loc 0
  while {[set loc [string first $substr $str $loc]] != -1} {
  lappend indices $loc
  incr loc
  }
  return $indices
  }
  
  Appears to be basically what I've been trying to do; give back the index of where the match is found, but then resume the search from that spot, not back at the beginning, until the entire string is searched. I tried incorporating this into my script, however it won't run as is (it hangs up because string first expects only two arguments and is getting three (string first $substr $str $loc). This is exactly what I'm trying to do, though...once I get the different index values in list format, it's no problem to then use those as markers to count back and pick out the slot numbers, etc.
  
  I tried using the string trimleft command with a wildcard as its char value to kind of comsume the string as it was being searched, so that the initial match was always wiped out, so that the next 'first' match would actually be the next match (if that makes any sense) until the end was reached, but the string trim commands don't seem to have the capability to trim from a certain index point within the string.
  
  Is there some other syntax that might work here?:
  
  while {[set loc [string first $substr $str $loc]] != -1}
  

 

[AviaTraining]

Okay, it appears as though you have an older version of Tcl that you're using that doesn't support the start index argument for the string first command. (I believe support for it was added in Tcl 8.2 or 8.2.1.) That's a bother, but we can code around it. Here's a version that should work for even relatively ancient versions of Tcl:

proc findAll {str substr} {
  set indices [list]

  # We'll use the length of the match string
  # later on in the algorithm

  set len [string length $substr]

  # "start" keeps track of our indices relative
  # to the start of the original string

  set start 0

  # "loc" keeps track of our index relative to
  # the start of our chopped string

  while {[set loc [string first $substr $str]] != -1} {

    # Add the relative index of the match to
    # our absolute index, then append the
    # absolute index to our list of matches

    incr start $loc
    lappend indices $start

    # Skip all of the characters in the current
    # match and truncate the string to search
    # accordingly

    incr loc $len
    incr start $len
    set str [string range $str $loc end]
  }
  return $indices
}

- Ken Jones, President, ken@avia-training.com
Avia Training and Consulting,

http://www.avia-training.com

866-TCL-HELP (866-825-4357) US Toll free
415-643-8692 Voice
415-643-8697 Fax

 

[rubberscissors]

Ahhh...you know I thought it was odd that I couldn't specify where in the string to start the search. It's my dusty old version of Tcl (it's the one packaged with Expect 5.21). That did the trick - that trick broke my scripting block and I've created a generic index/slot finder that this whole diag script can use. As I try and get this all under my belt you've been a huge help - thanks a million AviaTraining!