INTELLIGENT WORK FORUMS
FOR COMPUTER PROFESSIONALS
Come Join Us!
Are you a
Computer / IT professional?
Join Tek-Tips now!
- Talk With Other Members
- Be Notified Of Responses
To Your Posts
- Keyword Search
- One-Click Access To Your
Favorite Forums
- Automated Signatures
On Your Posts
- Best Of All, It's Free!
*Tek-Tips's functionality depends on members receiving e-mail. By joining you are opting in to receive e-mail.
Partner With Us!
"Best Of Breed" Forums Add Stickiness To Your Site
(Download This Button Today!)
Feedback
"...I have to add my thanks and appreciation for your wonderful site... People who frequent the site are the two best things - nice and smart..."
Geography
Where in the world do Tek-Tips members come from?
|
Newbie Question
|
|
Hi,
I have a folder with a list of files - 1.dat, 2.dat, 3.dat etc
The file structure is:
rndsid = |11026|1342296563|
lastactive = |1342296563|
lastvisit = |1340862364|
sitename = |Free PC Help|
email = | admin@mysite.com|
pmpopup = |1|
lastpost = |1340736611|
password = |******|
posts = |2|
registered = |1339327069|
dateformat = |0|
timezone = |0|
timeformat = |0|
sn = |Administrator|
siteurl = | http://www.pcfixzone.co.uk|
What I need to do is read the content of each .dat file and remove the entries for siteurl and sitename if they exist.
Is this possible?
Many thanks |
|
|
feherke (Programmer) |
15 Jul 12 5:22 |
Hi
Quote (martinjburgess)remove the entries for siteurl and sitename if they exist.
You mean, remove the entire such line ?
This does in-place modification, so better make backups before trying it :
CODE --> command-lineperl -pi -e 'undef$_ if/^site(url|name)\s*=/' *.dat Feherke.
http://feherke.github.com/ |
|
Thanks for the reply but that just gives a 500 error
What I need is a small script that when run will:
open each .dat file in the Members folder
Read each line
If siteurl or sitename exists either delete the line or set it to '' |
|
|
feherke (Programmer) |
15 Jul 12 6:06 |
Hi
Quote (martinjburgess)Thanks for the reply but that just gives a 500 error
What is " a 500 error" ? Show us how exactly you tried it and what exactly you got.
Feherke.
http://feherke.github.com/ |
|
I pasted it at the top of another script and when run it just gives a HTTP 500 error.
I think I will need something more on the lines of:
open(FILE,"$members/$_.dat");
while(<FILE>) {
code
code
code
close(FILE);
}
or something along those lines.
Sorry for being so vague. |
|
|
feherke (Programmer) |
15 Jul 12 6:48 |
Hi
Quote (martinjburgess)I pasted it at the top of another script
Quote (Feherke)vvvvvvvvvvv--- execute it from the command line CODE --> command-lineperl -pi -e 'undef$_ if/^site(url|name)\s*=/' *.dat
Quote (martinjburgess)just gives a HTTP 500 error.
How gets HTTP involved in all this ?
Anyway.
CODE --> Perluse File::Temp qw{tempfile};
foreach $file (<*.dat>) {
open FIN,"<$file";
($fout,$name)=tempfile;
while ($str=<FIN>) {
print $fout $str unless $str=~m/^site(url|name)\s*=/;
}
close $fout;
close FIN;
rename $name,$file;
}
Next time please specify all requirements at the beginning. Feherke.
http://feherke.github.com/ |
|
Sorry but no joy.
I did change this:
foreach $file (<*.dat>) {
to this:
foreach $file (<$members/*.dat>) {
as this is the folder the .dat files are in.
|
|
|
feherke (Programmer) |
15 Jul 12 8:27 |
|
Thanks for your patience.
This is the code and nothing happens:
#!/usr/bin/perl
use CGI::Carp fatalsToBrowser;
# Global information:
$theblahver = 16;
$version = $versioncr = '10.3.6'; # Said Version; Copyright version
$blanktarget = " target='_new'";
# Uncomment this for better time precision
use Time::HiRes qw(time);
# Filename information
$scriptname = $scriptname || 'Blah.pl'; # Change name of Blah.pl
$modrewrite = $modrewrite || '?'; # Setting, mod_rewrite: on = '' | off = '?'
use Fcntl ':flock';
# Default language
$languagep = "English";
$languages = "./Languages";
require('Settings.pl');
require("$code/QuickCore.pl");
UFS();
CheckCookies();
GetThemes();
$language = "$languages/$languagep";
require("$language.lng");
require("$code/Routines.pl");
require("$code/Load.pl");
# Remove the theme variable for guests/search engines
redirect() if($URL{'theme'} && $username eq 'Guest');
# Load basic features we can use later
CreateGroups();
BoardCheck();
ClickLog();
AL();
use File::Temp qw{tempfile};
foreach $file (<$members/*.dat>) {
open FIN,"<$file";
($fout,$name)=tempfile;
while ($str=<FIN>) {
print $fout $str unless $str=~m/^site(url|name)\s*=/;
}
close $fout;
close FIN;
rename $name,$file;
}
1; |
|
|
feherke (Programmer) |
15 Jul 12 9:01 |
Hi
So it is a CGI script. That makes the debugging abit complicated. The steps I would do :
- Check if the user with who's permission the web server runs, is allowed to write those files.
- Add warningsToBrowser :
CODE --> (fragment)# change this
use CGI::Carp qw{fatalsToBrowser warningsToBrowser};
# ...
# add this after the HTTP headers were sent
warningsToBrowser(1); - Check your web server's error log.
- Add some print calls to see what gets executed :
CODE --> (fragment)# ...
print "start dat editing <br>\n";
use File::Temp qw{tempfile};
print "directory is $members<br>\n";
print "files to process are ",<$members/*.dat>,"<br>\n";
foreach $file (<$members/*.dat>) {
print "start file $file<br>\n";
open FIN,"<$file";
($fout,$name)=tempfile;
print "write temp file $name<br>\n";
while ($str=<FIN>) {
print $fout $str unless $str=~m/^site(url|name)\s*=/;
}
close $fout;
close FIN;
print "rename file $name to $file<br>\n";
rename $name,$file;
print "end file $file<br>\n";
}
print "end dat editing<br>\n";
1;
Feherke.
http://feherke.github.com/ |
|
It looks like it is functioning but it doesn't actually delete the lines siteurl and sitename.
start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat./Members/5.dat
start file ./Members/1.dat
write temp file /tmp/WFTaSsK1ga
rename file /tmp/WFTaSsK1ga to ./Members/1.dat
end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/b5adpSuXC5
rename file /tmp/b5adpSuXC5 to ./Members/2.dat
end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/TSvi8giLoH
rename file /tmp/TSvi8giLoH to ./Members/3.dat
end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/CBfa12yMke
rename file /tmp/CBfa12yMke to ./Members/4.dat
end file ./Members/4.dat
start file ./Members/5.dat
write temp file /tmp/cOaQsiwo82
rename file /tmp/cOaQsiwo82 to ./Members/5.dat
end file ./Members/5.dat
end dat editing
Content-type: text/html; charset=UTF-8 |
|
|
feherke (Programmer) |
15 Jul 12 9:42 |
Hi
Seems my first point was right :
Quote (Feherke)- Check if the user with who's permission the web server runs, is allowed to write those files.
To be sure, you can add one more print :
CODE --> (fragment)print rename $name,$file;
It will output 1 on success and 0 on failure. Probably will be all 0s. This may be caused by more things, but most probably is the lack of write permission.
To correct this give write permission on Members/ directory. So from the command line first become superuser, then execute chmod a+x /path/to/Members/. ( Of course, in place of a ( all ) use g ( group ) in case the web server's configuration sets the group to the same group as Members/ is owned by. )
If still not works, you will have to add it to the files themselves with chmod a+x /path/to/Members/*.dat. But in this case you will have the same problem after creating/generating newer dat files. ( I suppose there will be more of them later. ) Feherke.
http://feherke.github.com/ |
|
Thanks, almost there.
It is now removing the line sitename but not the line siteurl
There are full write permissions on the folder. |
|
|
feherke (Programmer) |
15 Jul 12 10:35 |
Hi
Strange. Are you sure the sample in your opening post is correct ? I tested with that one and worked correctly. ( Given that siteurl is the last line, I double-checked both with and without trailing newline. )
Maybe there is a leading space in that line ? If yes, you can get ride of it with this :
CODE --> (fragment)print $fout $str unless $str=~m/^\s*site(url|name)\s*=/; Feherke.
http://feherke.github.com/ |
|
My mistake, it's not deleting any of the lines, it just reorderd them.
Here's what I now get:
start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat./Members/5.dat
start file ./Members/1.dat
write temp file /tmp/nKijKk93AQ
rename file /tmp/nKijKk93AQ to ./Members/1.dat
0end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/6ydGcqERS1
rename file /tmp/6ydGcqERS1 to ./Members/2.dat
0end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/Ck7RE8hTNL
rename file /tmp/Ck7RE8hTNL to ./Members/3.dat
0end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/rSsiIt7fF0
rename file /tmp/rSsiIt7fF0 to ./Members/4.dat
0end file ./Members/4.dat
start file ./Members/5.dat
write temp file /tmp/px78Uqg55x
rename file /tmp/px78Uqg55x to ./Members/5.dat
0end file ./Members/5.dat
end dat editing
|
|
|
feherke (Programmer) |
15 Jul 12 10:57 |
Hi
Well, those 0s in the output are clearly showing that rename is still failing.
There is another step to include to see the exact reason of the failure :
CODE --> (fragment)print "rename file $name to $file<br>\n";
print rename $name,$file;
print "rename error $!<br>\n";
undef $!;
print "end file $file<br>\n"; Feherke.
http://feherke.github.com/ |
|
start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat
start file ./Members/1.dat
write temp file /tmp/08yw9L8QjX
rename file /tmp/08yw9L8QjX to ./Members/1.dat
0rename error Invalid cross-device link
end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/DXfbMJ81WL
rename file /tmp/DXfbMJ81WL to ./Members/2.dat
0rename error Invalid cross-device link
end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/FKbrf0HXol
rename file /tmp/FKbrf0HXol to ./Members/3.dat
0rename error Invalid cross-device link
end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/iSihH4mK3P
rename file /tmp/iSihH4mK3P to ./Members/4.dat
0rename error Invalid cross-device link
end file ./Members/4.dat
end dat editing
|
|
|
feherke (Programmer) |
15 Jul 12 11:42 |
Hi
Ah. Got it. Your Members/ and /tmp/ directories are on different partitions. Then something stronger than the basic rename is needed :
CODE --> Perluse File::Temp qw{tempfile};
use File::Copy;
foreach $file (<*.dat>) {
open FIN,"<$file";
($fout,$name)=tempfile;
while ($str=<FIN>) {
print $fout $str unless $str=~m/^site(url|name)\s*=/;
}
close $fout;
close FIN;
move $name,$file;
} Feherke.
http://feherke.github.com/ |
|
You are a genius Sir, thank you very much, all working like a charm. |
|
One last question.
Is it possible to skip if the user is administrator?
This is the line in the .dat file
sn = |Administrator|
Cheers |
|
|
feherke (Programmer) |
15 Jul 12 12:18 |
Hi
What do you mean by " skip" ? Remove the siteurl and sitename lines only if the sn's value is not "Administrator" ? Feherke.
http://feherke.github.com/ |
|
That's correct, if sn's value is administrator then leave the siteurl and sitename intact. |
|
|
feherke (Programmer) |
15 Jul 12 12:48 |
Hi
Well, this can be essentially done in two different ways :
- Single pass : Process the file as now, but if sn = Administrator line is met, cancel the processing.
- Two passes : Read the file and check the existence of sn = Administrator line. If non find, process the file as now.
Probably I should have asked this earlier. The dat files will be all like that sample, with 10~20 lines, under 1Kb size ? If yes, we can just slurp in the entire file instead of processing line by line. This way neither temporary file and file moving is involved :
CODE --> Perlundef $/;
foreach $file (<*.dat>) {
open FIL,"<$file";
$data=<FIL>;
close FIL;
next if $data=~m/^sn\s*=\s*\|Administrator\|/mi;
$data=~s/^site(url|name)\s*=.*\r?\n?//gm;
open FIL,">$file";
print FIL $data;
close FIL;
} Feherke.
http://feherke.github.com/ |
|
That last code completely mucks up the table layout of my test forum for some reason.
These dat files are only small, never seen one larger than 1Kb |
|
|
feherke (Programmer) |
15 Jul 12 13:43 |
Hi
I assume is because my ugly habit to unset the $/. This should play nicely :
CODE --> Perl#undef $/; # <-- remove this one
foreach $file (<*.dat>) {
local $/;
# ...
} Feherke.
http://feherke.github.com/ |
|
That's done the trick, I'll do some more testing and let you know.
Thank you, much appreciated. |
|
Sorry it's late but just want to say thanks to feherke for his patience and knowledge. Everything I wanted to achieve I have.
Thank You |
|
Nothing says "Thank you" better than a star...  |
|
|
 |
|
Talk To Other Members
Read More Threads Like This One