REGEX multiple matches in one line

[NetworkGhost]

Made a script to get google results. When I parse the file it seems google does not include carriage returns. What I need to do is regex all the links from the file. If there are multiple matches in one line then I need to pull all occurrences of the string.

so this is what I was working with:

regexp {a.*craig[^>]*>(.*?)</a>} $line url1

I have tried a few things like outputing to more variables, doesnt seem to work. Can regex do this for me? Any help is appreciated. the other option I thought would be to split the results at every hyperlink. Will try that next.

http://www.wr-mem.com

 

[Bong]

I'm sure you can form a regular expression that does what you want. What I don't know is what you want. What do you want regexp to find? That is, what are the characteristics of the strings you're looking for?

_________________
Bob Rashkin

 

[NetworkGhost]

What I did was a wget for a google string. The information returned doesnt have carriage returns so I get a big line like below:

<a href="

http://blogsearch.google.com/blogse...ficial&q=Cisco+ASA+cpg+2008-03-02+site:craigs

list.org&um=1&ie=UTF-8&sa=N&tab=wb">Blogs</a></span> <span class=gb2><div></div></a></span> <span class=gb2><a href="

http://www.youtube.com/results?hl=en&clie

nt=firefox-a&rls=org.mozilla:en-USfficial&q=Cisco+ASA+cpg+2008-03-02+site:craigslist.org&um=1&ie=UTF-8&sa=N&tab=w1">YouTube</a></span> <span class=gb2><a hr
ef="

http://www.google.com/calendar/render?um=1&ie=UTF-8&sa=N&tab=wc">Calendar</a></span>

<span class=gb2><a href="

http://picasaweb.google.com/lh/searchbrowse?

hl=en&client=firefox-a&rls=org.mozilla:en-USfficial&q=Cisco+ASA+cpg+2008-03-02+site:craigslist.org&um=1&ie=UTF-8&sa=N&tab=wq">Photos</a></span> <span class=
gb2><a href="

http://docs.google.com/?um=1&ie=UTF-8&sa=N&tab=wo">Documents</a></span>

<span class=gb2><a href="

http://www.google.com/reader/view/?hl=en&client=

firefox-a&rls=org.mozilla:en-USfficial&q=Cisco+ASA+cpg+2008-03-02+site:craigslist.org&um=1&ie=UTF-8&sa=N&tab=wy">Reader</a></span> <span class=gb2><div></di
v></a></span> <span class=gb2><a href="

http://www.google.com/intl/en/options/">even

more &raquo;</a></span> </nobr></div><div class=gbh style=left:0></div><di
v class=gbh style=right:0></div><div align=right id=guser style="font-size:84%;padding:0 0 4px" width=100%><nobr><a href="

https://www.google.com/accounts/Logi

n?continue=

http://www.google.com/search?hl=en&...S%3Aofficial&q=Cisco+ASA+cpg+2008-03-02+site%

253Acraigslist.org%26btnG%3DSearch&hl=en">Sign in</a></nobr></div><table class=tb style=clear:left width=100%><tr><form name=gs method=GET action=/search><td
class=tc valign=top><a id=logo href="

http://www.google.com/webhp?hl=en"

title="Go to Google Home">Google<span></span></a></td><td style="padding:0 0 7px;paddi
ng-left:8px" valign=top width=100%><table class=tb style=margin-top:25px><tr><td class=tc nowrap><input type=hidden name=hl value="en"><input type=hidden nam
e=client value="firefox-a"><input type=hidden name=rls value="org.mozilla:en-USfficial"><input type=hidden name=hs value="4uf"><input type=text name=q size=
49 maxlength=2048 value="Cisco ASA cpg 2008-03-02 site:craigslist.org" title="Search"> <input type=submit name="btnG" value="Search"></td><td class=tc nowrap
width=100%><span id=ap>&nbsp;&nbsp;<a href=/advanced_search?q=Cisco+ASA+cpg+2008-03-02+site:craigslist.org&hl=en&client=firefox-a&rls=org.mozilla:en-USffici
al&hs=4uf>Advanced Search</a><br>&nbsp; <a href=/preferences?q=Cisco+ASA+cpg+2008-03-02+site:craigslist.org&hl=en&client=firefox-a&rls=org.mozilla:en-USffic
ial&hs=4uf>Preferences</a></span></td></tr></table></td></tr></form></table><table border=0 cellpadding=0 cellspacing=0 width=100% class="t bt"><tr><td nowrap
><span id=sd>&nbsp;Web&nbsp;</span></td><td align=right nowrap><font size=-1>Results <b>1</b> - <b>1</b> of <b>1</b> from <b>craigslist.org</b> for <b>Cisco A
SA cpg 2008-03-02</b>. (<b>0.08</b> seconds)&nbsp;</font></td></tr></table> <div id=res><!--a--><div><div class=g><!--m--><h2 class=r><a href="

http://sfbay.

craigslist.org/sby/cpg/593782938.html" class=l onmousedown="return clk(this.href,'','','res','1','')">need help setting up a <b>cisco asa</b> 5505</a><


What I want to do is pull all hyperlinks from that line and then create a list.

http://www.wr-mem.com

 

[Bong]

So let's say you want a list of strings, each element of which is the substring lying between "<a href=" and ">", right?

I'm really no expert in regular expressions (far from it) so maybe one or another will pipe in.

Let's say you have a string:
<a href="abc.123.bbb all 4"> lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4">
lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4"> lkjlkj; aii3 aldjkl;j;<a href
="abc.123.bbb all 4"> lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4"> lkjlkj;
aii3 aldjkl;j;<a href="abc.123.bbb all 4"> lkjlkj; aii3 aldjkl;j;<a href="abc.1
23.bbb all 4"> lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4"> lkjlkj; aii3 a
ldjkl;j;

If we do the following substitution:
regsub -all {(a href=")([^"]*)(\">)} $a "&\n" b
then "b" will now be:
[tt]<a href="abc.123.bbb all 4">
lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4">
lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4">
lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4">
lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4">
lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4">
lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4">
lkjlkj; aii3 aldjkl;j;<a href="abc.123.bbb all 4">
lkjlkj; aii3 aldjkl;j;[/tt]
Not quite what you want but there's now one link per line.
Now for each line (foreach line $b {, we can get only from "<" to ">":
set i1 [string first < $line]
set i2 [string first < $line]
set subline [string range $line $i1 $i2]

Now you can do the same thing with the quotes:
set i3 [string first {="} $subline]
set i4 [string first {">} $subline]
incr i3 2;#to get past the quote
incr i4 -2;#to stop before the quote
set subline2 [string range $subline $i3 $i4]



_________________
Bob Rashkin

 

[NetworkGhost]

Thanks. Will check out today and report back. Appreciate the help.

http://www.wr-mem.com

 

[geirendre]

If you have egrep available you can also try something like this:

egrep -o 'href=.*?' /path/to/linkfile

and see if it does what you want.

HTH

 

[Bong]

This may be fewer steps (maybe not).
1. strip off anything before the first link
2. build the regular expression to find the links (between the quotes)
3. using regsub, get those links only, separated by "|" (I can't get it to insert the linefeed).
4. split that string on "|"
5. join that list with "\n"

starting with a string, a:
aaa <a href="xyz.abc.com/fgh ?abc">1a2b 3c4d</a> g6j7k9aaa <a href="xyz.abc.com/
fgh ?abc">1a2b 3c4d</a> g6j7k9aaa <a href="xyz.abc.com/fgh ?abc">1a2b 3c4d</a> g
6j7k9aaa <a href="xyz.abc.com/fgh ?abc">1a2b 3c4d</a> g6j7k9

regsub {[^<]*} $a {} b; #1
set re {(<a href=\")([^\"]*)(\">[^<]*</a>[^<]*)}; #2
regsub -all $re $b {\2 |} c; #3
set lstc [split $c |]; #4
set strc [join $lstc \n]

after #1, b is now:
<a href="xyz.abc.com/fgh ?abc">1a2b 3c4d</a> g6j7k9aaa <a href="xyz.abc.com/fgh
?abc">1a2b 3c4d</a> g6j7k9aaa <a href="xyz.abc.com/fgh ?abc">1a2b 3c4d</a> g6j7k
9aaa <a href="xyz.abc.com/fgh ?abc">1a2b 3c4d</a> g6j7k9

after #3, c is now:
xyz.abc.com/fgh ?abc |xyz.abc.com/fgh ?abc |xyz.abc.com/fgh ?abc |xyz.abc.com/fg
h ?abc |

after #4, lstc is now:
{xyz.abc.com/fgh ?abc } {xyz.abc.com/fgh ?abc } {xyz.abc.com/fgh ?abc } {xyz.abc
.com/fgh ?abc } {}

after #5, strc is now:
xyz.abc.com/fgh ?abc
xyz.abc.com/fgh ?abc
xyz.abc.com/fgh ?abc
xyz.abc.com/fgh ?abc

_________________
Bob Rashkin

 

[Bong]

Obviously steps 4 and 5 can be combined:
set strc [join [split $c |] \n]

_________________
Bob Rashkin

 

[prex1]

This does what you seem to be looking for

set mylinks [regexp -all -inline {<a .+?>(.*?)</a>} $line]
foreach {tag cont} $mylinks {
  if [string first craig $tag]>0 {
    ... process $cont here ...
  }
}

That is it returns the content of the hyperlinks containing 'craig' in the whole hyperlink. You need to extract first all the links, then filter out those that do not contain the searched string: can't see a way of obtaining directly only those.
Note that the list [tt]mylinks[/tt] will inevitably contain also the full matched expressions, so the contents of the hyperlink occupy every 2nd item in the list (just to be remembered when parsing the list, but also useful for filtering).
Note that your string seem to be malformed: there is at least a [tt]</a>[/tt] alone.

Franco

http://www.xcalcs.com

: Online engineering calculations

http://www.megamag.it

: Magnetic brakes for fun rides

http://www.levitans.com

: Air bearing pads