Need awk script for removing duplicate records

[Rastamed]

I have log file having Traffic line

Code:

2011-05-21 15:11:50.356599 TCP (6), length: 52) 10.10.10.1.3020 > 10.10.10.254.50404:
2011-05-21 15:11:50.652739 TCP (6), length: 52) 10.10.10.254.50404 > 10.10.10.1.3020:
2011-05-21 15:11:50.652558 TCP (6), length: 89) 10.10.10.1.3020 > 10.10.10.254.50404:
2011-05-21 15:11:50.852325 TCP (6), length: 32) 10.10.10.1.3020 > 10.10.10.254.50404:


the idea is to remove the lines that are repeated more than once , write how many times the line is repeated and the summation field length . I also want to arrange fields to have the following matches

Code:

2011-05-21 15:11:50.356599 TCP (6) length 141 10.10.10.1 3020 > 10.10.10.254 50404 3
2011-05-21 15:11:50.652739 TCP (6) length 52 10.10.10.254 50404 > 10.10.10.1 3020 1


I managed to get this result but it is not enough

Code:

awk '{x[substr ($0,28)]++;y[substr ($0,28)]=$2} END { for (i in x) printf "%s %d\n",yi,x}' file.txt




Code:

15:11:50.356599 TCP (6), length: 52 10.10.10.1.3020 > 10.10.10.254.50404 3
15:11:50.652739 TCP (6), length: 52 10.10.10.254.50404 > 10.10.10.1.3020 1

 

[Annihilannic]

How about this slight variation of your solution:

awk '{x[[blue]$7[/blue],[blue]$9[/blue]]++;y[[blue]$7[/blue],[blue]$9[/blue]]=[blue]$0[/blue]} [green]END[/green] { [olive]for[/olive] (i [olive]in[/olive] x) [b]printf[/b] [red]"[/red][purple]%s %d\n[/purple][red]"[/red],y[i],x[i]}' file.txt

I'm assuming that you're only interested in lines where the source and destination IP/port combinations match.

Annihilannic.