Moveing emails to different directories based of email date 2

[CharlieMike73]

Hi, I have a directory on a Linux box that has over 1.5Gig of emails that I need to move into directories grouped by the year, then a sub directory for each quarter (Jan-Mar in 1st Quarter, Apr-Jun into 2nd of given year, and so on).

When I tried to do this using the web-based email interfaces I would get an ‘out of resources’ error due to trying to move the large volume of emails. I do have SSH access and this is where I would expect to run the script/command.

Any help is appreciated, and I do apologies if this truly is as simple as it first appears; I just need someone who knows the command(s) and experience with Linux to help.

 

[Annihilannic]

What format are the emails in? One file per email? An obfuscated example of one of the files in question would help.

Annihilannic.

 

[CharlieMike73]

Sorry, I have been out of the office and unable to check this forum... anyway each email is an individual file in the directory like so.

8 -rw-rw---- 1 {security user group settings} 4473 Jan 10 2008 1273506657.H459901P14287.anglo.my-domain.com,S\=4473:2,
8 -rw-rw---- 1 {security user group settings} 4403 Feb 10 2009 1273506657.H969365P14301.anglo.my-domain.com,S\=4403:2,
8 -rw-rw---- 1 {security user group settings} 4535 May 11 2009 1273577751.H392754P28280.anglo.my-domain.com,S\=4535:2,
8 -rw-rw---- 1 {security user group settings} 4366 May 11 07:35 1273577759.H9348P29771.anglo.my-domain.com,S\=4366:2,
8 -rw-rw---- 1 {security user group settings} 4363 May 12 00:05 1273637129.H534018P19865.anglo.my-domain.com,S\=4363:2,
8 -rw-rw---- 1 {security user group settings} 4452 May 12 00:05 1273637130.H38771P19873.anglo.my-domain.com,S\=4452:2,
8 -rw-rw---- 1 {security user group settings} 4744 May 22 04:25 1274516747.H299499P26257.anglo.my-domain.com,S\=4744:2,
4 -rw-rw---- 1 {security user group settings} 2523 Jul 6 07:07 1278414451.H804998P30270.anglo.my-domain.com,S\=2523:2,S
4 -rw-rw---- 1 {security user group settings} 2499 Jul 11 14:44 1278873862.H76864P6139.anglo.my-domain.com,S\=2499:2,S

I did have some success using 'find' and some strange setting to calculate the dates backwards from today but that required me to try and figure the correct backdate values and it didnt move the files which is also what I need to do, and if I understand correctly 'find' also calculated back using the specific time it was run, so the results an hour later (or even a minute later for that matter) may be different that the initial result. Using the date the file was created or last modified is sufficient as this is near always the date the email was sent.

Again, thank you for your time and efforts to help me resolve this.

Regards,
Charlie

 

[PHV]

A starting point:

getFileDate() {
eval $(perl -e '
($ss,$nn,$hh,$dd,$mm,$yy,$wd,$yd,$dst)=localtime((stat("'$1'"))[9]);
printf("y=%d q=%d",$yy+1900,1+($mm/3));
')
}
destdir='/path/to/dest'
cd '/path/to/mailbox'
ls | while read f; do
  [ -f "$f" ] || continue
  getFileDate "$f"
  d="$destdir/$y/$q"
  [ -d "$d" ] || mkdir -p "$d"
  mv "$f" "$d"
done

Hope This Helps, PH.
FAQ219-2884
FAQ181-2886

 

[Annihilannic]

Pinching PHVs idea but making it a bit more perl-ey:

[gray]#!/usr/bin/perl -w[/gray]

[url=http://perldoc.perl.org/functions/my.html][black][b]my[/b][/black][/url] [blue]$destdir[/blue]=[red]'[/red][purple]/dest[/purple][red]'[/red][red];[/red]
[black][b]my[/b][/black] [blue]$srcdir[/blue]=[red]'[/red][purple]/src[/purple][red]'[/red][red];[/red]

[olive][b]foreach[/b][/olive] [red]([/red][url=http://perldoc.perl.org/functions/glob.html][black][b]glob[/b][/black][/url] [red]"[/red][purple][blue]$srcdir[/blue]/*[/purple][red]"[/red][red])[/red] [red]{[/red]
        [black][b]my[/b][/black] [red]([/red][blue]$ss[/blue],[blue]$nn[/blue],[blue]$hh[/blue],[blue]$dd[/blue],[blue]$mm[/blue],[blue]$yy[/blue],[blue]$wd[/blue],[blue]$yd[/blue],[blue]$dst[/blue][red])[/red] = [url=http://perldoc.perl.org/functions/localtime.html][black][b]localtime[/b][/black][/url][red]([/red][red]([/red][url=http://perldoc.perl.org/functions/stat.html][black][b]stat[/b][/black][/url][red]([/red][blue]$_[/blue][red])[/red][red])[/red][red][[/red][fuchsia]9[/fuchsia][red]][/red][red])[/red][red];[/red]
        [black][b]my[/b][/black] [blue]$dest[/blue] = [url=http://perldoc.perl.org/functions/sprintf.html][black][b]sprintf[/b][/black][/url][red]([/red][red]"[/red][purple][blue]$destdir[/blue]/%d/%d[/purple][red]"[/red],[blue]$yy[/blue]+[fuchsia]1900[/fuchsia],[fuchsia]1[/fuchsia]+[red]([/red][blue]$mm[/blue]/[fuchsia]3[/fuchsia][red])[/red][red])[/red][red];[/red]
        [olive][b]if[/b][/olive] [red]([/red]! [url=http://perldoc.perl.org/functions/-X.html][black][b]-d[/b][/black][/url] [red]"[/red][purple][blue]$dest[/blue][/purple][red]"[/red][red])[/red] [red]{[/red] [url=http://perldoc.perl.org/functions/system.html][black][b]system[/b][/black][/url] [red]"[/red][purple]echo mkdir -p [blue]$dest[/blue][purple][b]\n[/b][/purple][/purple][red]"[/red][red];[/red] [red]}[/red]
        [black][b]system[/b][/black] [red]"[/red][purple]echo mv [blue]$_[/blue] [blue]$dest[/blue][purple][b]\n[/b][/purple][/purple][red]"[/red][red];[/red]
[red]}[/red]

You can take out the two echos if you're satisfied that it's going to do what you require.

Also, if performance is a consideration and you aren't going to be moving files across filesystem boundaries, consider replacing the second system call with the built-in rename function.

Annihilannic.

 

[CharlieMike73]

Thank you both for your scripts, I created a shell script using VIM but when I configured the destination directory I did it wrong and now need a way to get all the files back into a single directory so I can run the script again, but with the correct new location; more accuratly the naming convention I used for the destination should have been like '.yyyy.qQ' (e.g. .2010.1Q for 1st Quarter of 2010 note the prefixed period)

I also noticed that when it moved the files it changed the user/group ownership; so where should I put a CHOWN command to revert them back to their original owner - it is always the same owner so no logic is needed to determin the owner before moving.

Between the 'mv' and 'done' lines?

Lastly, if I wanted to go one step further and sort the files out by month, how would I need modify PHV's date string

 

[Annihilannic]

It's strange that the ownership was changed; you definitely mv'd them rather than cp'd them, right?

Assuming they are all in subdirectories under /the/wrong/place, you could could correct the ownership using:

chown -R username /the/wrong/place

and then move them back in one step using:

find /the/wrong/place -type f -exec mv {} /the/original/directory \;

Both of these will affect *all* files under /the/wrong/place, so if you have other stuff under there you'll need to be more strategic.

To adjust either script to use months instead of quarters, just change 1+($mm/3) to 1+$mm.

Annihilannic.