Newbie Question - Working with Arrays

[lmbylsma]

I'm trying to get started on some basic processing of data in array form. Basically this is a segment of what the data file looks like:

20070314180137 1003 1 641 1
20070314180137 1003 3 598 9
20070314180137 1003 4 2466 12
20070314180137 1003 101 610 32
20070314180137 1003 102 410 41
20070312141422 1003 -32767 0 9999
20070312163759 1003 1 426 2
20070312163759 1003 101 352 69
20070312163759 1003 102 189 74
20070312163759 1003 103 273 50
20070312163759 1003 104 351 87

What I want it to do is process all the lines of data except the ones with the lines with the 9999 in it. Then I want it to organize it to put the date/time in a more readable format and organize it like this so that all the variables from the same date are on the same line:

1003 3/14/2007 18:01:37 1 641 1 3 598 9 101 610 32 102 410 41
1003 3/12/2007 16:37:59 1 426 2 101 352 69 102 189 74 103 273 50 104 351 87

etc..

Any help on getting me started with this would be greatly appreciated.


Thanks,


Lauren

 

[spookie]

Hi,
To start with here is how you can do
Read the data line by line and check using regex if it does not have pattern '9999'.

Then split it with spaces and strip out date (first eight charactres). Form a hash of the dates with key as dates and value as rest of the lines and each time check if the date element exists in hash. If yes append the value.

Print the hash in format you want.

--------------------------------------------------------------------------
I never set a goal because u never know whats going to happen tommorow.

 

[KevinADC]

What have you tried so far Lauren? Do you know any perl? Do you know how to create/use complex data such as a hash of arrays?

------------------------------------------
- Kevin, perl coder unexceptional!

 

[lmbylsma]

I was trying to do this using just basic arrays which I am more familiar with but it seems to be more complicated to try to do it that way. I haven't tried using hashes yet, I'm pretty new to perl, I only use it every now and then to do some things with data that excel and my stats program can't do properly. I will try to figure it out using a hash table as suggested, but if anyone wants to post an example of what the code would look at for creating the hash that would be really helpful.

Thanks,

Lauren

 

[KevinADC]

Here is a generic example to get you started:

[url=http://perldoc.perl.org/functions/use.html][black][b]use[/b][/black][/url] [green]Data::Dumper[/green][red];[/red]
[url=http://perldoc.perl.org/functions/my.html][black][b]my[/b][/black][/url] [blue]%HoA[/blue] = [red]([/red][red])[/red][red];[/red]

[olive][b]while[/b][/olive][red]([/red]<DATA>[red])[/red][red]{[/red]
   [olive][b]next[/b][/olive] [olive][b]if[/b][/olive] [red]([/red][red]/[/red][purple]9999$[/purple][red]/[/red][red])[/red][red];[/red]
   [url=http://perldoc.perl.org/functions/chomp.html][black][b]chomp[/b][/black][/url][red];[/red]
   [black][b]my[/b][/black] [blue]@t[/blue] = [url=http://perldoc.perl.org/functions/split.html][black][b]split[/b][/black][/url][red]([/red][red]/[/red][purple][purple][b]\s[/b][/purple]+[/purple][red]/[/red][red])[/red][red];[/red]
   [url=http://perldoc.perl.org/functions/push.html][black][b]push[/b][/black][/url] [blue]@[/blue][red]{[/red] [blue]$HoA[/blue][red]{[/red][blue]$t[/blue][red][[/red][fuchsia]1[/fuchsia][red]][/red][red]}[/red][red]{[/red][blue]$t[/blue][red][[/red][fuchsia]0[/fuchsia][red]][/red][red]}[/red] [red]}[/red], [red][[/red] [blue]@t[/blue][red][[/red][fuchsia]2[/fuchsia],[fuchsia]3[/fuchsia],[fuchsia]4[/fuchsia][red]][/red] [red]][/red][red];[/red] 
[red]}[/red]
[url=http://perldoc.perl.org/functions/print.html][black][b]print[/b][/black][/url] Dumper \[blue]%HoA[/blue][red];[/red]

[teal]__DATA__[/teal]
[teal]20070314180137      1003         1            641         1[/teal]
[teal]20070314180137      1003         3            598         9[/teal]
[teal]20070314180137      1003         4           2466        12[/teal]
[teal]20070314180137      1003       101            610        32[/teal]
[teal]20070314180137      1003       102            410        41[/teal]
[teal]20070312141422      1003    -32767              0      9999[/teal]
[teal]20070312163759      1003         1            426         2[/teal]
[teal]20070312163759      1003       101            352        69[/teal]
[teal]20070312163759      1003       102            189        74[/teal]
[teal]20070312163759      1003       103            273        50[/teal]
[teal]20070312163759      1003       104            351        87[/teal]

[tt]------------------------------------------------------------
Core (perl 5.10.0) Modules used :
[ul]
[li]Data:umper - stringified perl data structures, suitable for both printing and eval[/li]
[/ul]
[/tt]

10 people will write that 10 different ways. There is no validation of the data either. This shows the basic concept of creating a hash of arrays, which for all I know is not even what you want, but it seems like it would work judging by your example output. This tutorial may also help you:

http://perldoc.perl.org/perldsc.html

------------------------------------------
- Kevin, perl coder unexceptional!

 

[lmbylsma]

Thanks, that is very helpful to get me started.

 

[MillerH]

Here is a slightly more expanded version of what you're aiming to get. This does the grouping and filtering. The only thing it doesn't do yet is the reformatting of the date.

[url=http://perldoc.perl.org/functions/my.html][black][b]my[/b][/black][/url] [blue]$last_datetime[/blue] = [red]'[/red][purple][/purple][red]'[/red][red];[/red]
[black][b]my[/b][/black] [blue]$last_id[/blue] = [red]'[/red][purple][/purple][red]'[/red][red];[/red]

[olive][b]while[/b][/olive] [red]([/red]<DATA>[red])[/red] [red]{[/red]
	[url=http://perldoc.perl.org/functions/chomp.html][black][b]chomp[/b][/black][/url][red];[/red]
	
	[black][b]my[/b][/black] [red]([/red][blue]$datetime[/blue], [blue]$id[/blue], [blue]@values[/blue][red])[/red] = [url=http://perldoc.perl.org/functions/split.html][black][b]split[/b][/black][/url] [red]/[/red][purple][purple][b]\s[/b][/purple]+[/purple][red]/[/red][red];[/red]
	
	[gray][i]# Skipping Data with 9999[/i][/gray]
	[olive][b]if[/b][/olive] [red]([/red][url=http://perldoc.perl.org/functions/grep.html][black][b]grep[/b][/black][/url] [red]{[/red][blue]$_[/blue] eq [fuchsia]9999[/fuchsia][red]}[/red] [blue]@values[/blue][red])[/red] [red]{[/red]
		[olive][b]next[/b][/olive][red];[/red]
	[red]}[/red]
	
	[gray][i]# Displaying Header Info, if new[/i][/gray]
	[olive][b]if[/b][/olive] [red]([/red][blue]$last_datetime[/blue] ne [blue]$datetime[/blue] || [blue]$last_id[/blue] ne [blue]$id[/blue][red])[/red] [red]{[/red]
		[url=http://perldoc.perl.org/functions/print.html][black][b]print[/b][/black][/url] [red]"[/red][purple][purple][b]\n[/b][/purple][/purple][red]"[/red] [olive][b]if[/b][/olive] [blue]$last_datetime[/blue][red];[/red]
		
		[blue]$last_datetime[/blue] = [blue]$datetime[/blue][red];[/red]
		[blue]$last_id[/blue] = [blue]$id[/blue][red];[/red]
		
		[black][b]print[/b][/black] [red]"[/red][purple][blue]$id[/blue] [blue]$datetime[/blue][/purple][red]"[/red][red];[/red]
	[red]}[/red]
	
	[gray][i]# Append values to list[/i][/gray]
	[olive][b]foreach[/b][/olive] [red]([/red][blue]@values[/blue][red])[/red] [red]{[/red]
		[black][b]print[/b][/black] [red]"[/red][purple] [blue]$_[/blue][/purple][red]"[/red][red];[/red]
	[red]}[/red]
[red]}[/red]

[black][b]print[/b][/black] [red]"[/red][purple][purple][b]\n[/b][/purple][/purple][red]"[/red][red];[/red]

[teal]__DATA__[/teal]
[teal]20070314180137      1003         1            641         1[/teal]
[teal]20070314180137      1003         3            598         9[/teal]
[teal]20070314180137      1003         4           2466        12[/teal]
[teal]20070314180137      1003       101            610        32[/teal]
[teal]20070314180137      1003       102            410        41[/teal]
[teal]20070312141422      1003    -32767              0      9999[/teal]
[teal]20070312163759      1003         1            426         2[/teal]
[teal]20070312163759      1003       101            352        69[/teal]
[teal]20070312163759      1003       102            189        74[/teal]
[teal]20070312163759      1003       103            273        50[/teal]
[teal]20070312163759      1003       104            351        87[/teal]

- Miller