Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations TouchToneTommy on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Newbie Question 1

Status
Not open for further replies.

martinjburgess

IS-IT--Management
Jun 8, 2012
21
GB
Hi,

I have a folder with a list of files - 1.dat, 2.dat, 3.dat etc
The file structure is:

rndsid = |11026|1342296563|
lastactive = |1342296563|
lastvisit = |1340862364|
sitename = |Free PC Help|
email = |admin@mysite.com|
pmpopup = |1|
lastpost = |1340736611|
password = |******|
posts = |2|
registered = |1339327069|
dateformat = |0|
timezone = |0|
timeformat = |0|
sn = |Administrator|
siteurl = |
What I need to do is read the content of each .dat file and remove the entries for siteurl and sitename if they exist.

Is this possible?

Many thanks
 
Hi

martinjburgess said:
remove the entries for siteurl and sitename if they exist.
You mean, remove the entire such line ?

This does in-place modification, so better make backups before trying it :
Code:
perl -pi -e '[b]undef[/b][navy]$_[/navy] [b]if[/b][green][i]/^site(url|name)\s*=/[/i][/green]' *.dat


Feherke.
[link feherke.github.com/][/url]
 
Thanks for the reply but that just gives a 500 error

What I need is a small script that when run will:

open each .dat file in the Members folder
Read each line
If siteurl or sitename exists either delete the line or set it to ''
 
I pasted it at the top of another script and when run it just gives a HTTP 500 error.

I think I will need something more on the lines of:

open(FILE,"$members/$_.dat");
while(<FILE>) {
code
code
code
close(FILE);
}

or something along those lines.

Sorry for being so vague.
 
Hi

martinjburgess said:
I pasted it at the top of another script
Feherke said:
[red]vvvvvvvvvvv--- execute it from the command line[/red]​
Code:
perl -pi -e '[b]undef[/b][navy]$_[/navy] [b]if[/b][green][i]/^site(url|name)\s*=/[/i][/green]' *.dat

martinjburgess said:
just gives a HTTP 500 error.
How gets HTTP involved in all this ?

Anyway.
Perl:
[b]use[/b] File[teal]::[/teal]Temp [b]qw[/b][teal]{[/teal]tempfile[teal]}[/teal][teal];[/teal]

[b]foreach[/b] [navy]$file[/navy] [teal]([/teal][green][i]<*.dat>[/i][/green][teal])[/teal] [teal]{[/teal]
  [b]open[/b] FIN[teal],[/teal][green][i]"<$file"[/i][/green][teal];[/teal]
  [teal]([/teal][navy]$fout[/navy][teal],[/teal][navy]$name[/navy][teal])=[/teal]tempfile[teal];[/teal]
  [b]while[/b] [teal]([/teal][navy]$str[/navy][teal]=[/teal][green][i]<FIN>[/i][/green][teal])[/teal] [teal]{[/teal]
    [b]print[/b] [navy]$fout[/navy] [navy]$str[/navy] [b]unless[/b] [navy]$str[/navy][teal]=~[/teal][b]m[/b][fuchsia]/^site(url|name)\s*=/[/fuchsia][teal];[/teal]
  [teal]}[/teal]
  [b]close[/b] [navy]$fout[/navy][teal];[/teal]
  [b]close[/b] FIN[teal];[/teal]

  [b]rename[/b] [navy]$name[/navy][teal],[/teal][navy]$file[/navy][teal];[/teal]
[teal]}[/teal]
Next time please specify all requirements at the beginning.

Feherke.
[link feherke.github.com/][/url]
 
Sorry but no joy.

I did change this:

foreach $file (<*.dat>) {

to this:

foreach $file (<$members/*.dat>) {

as this is the folder the .dat files are in.
 
Thanks for your patience.
This is the code and nothing happens:

#!/usr/bin/perl

use CGI::Carp fatalsToBrowser;

# Global information:
$theblahver = 16;
$version = $versioncr = '10.3.6'; # Said Version; Copyright version
$blanktarget = " target='_new'";

# Uncomment this for better time precision
use Time::HiRes qw(time);

# Filename information
$scriptname = $scriptname || 'Blah.pl'; # Change name of Blah.pl
$modrewrite = $modrewrite || '?'; # Setting, mod_rewrite: on = '' | off = '?'

use Fcntl ':flock';

# Default language
$languagep = "English";
$languages = "./Languages";

require('Settings.pl');

require("$code/QuickCore.pl");
UFS();
CheckCookies();
GetThemes();

$language = "$languages/$languagep";
require("$language.lng");
require("$code/Routines.pl");
require("$code/Load.pl");


# Remove the theme variable for guests/search engines
redirect() if($URL{'theme'} && $username eq 'Guest');

# Load basic features we can use later
CreateGroups();
BoardCheck();
ClickLog();
AL();

use File::Temp qw{tempfile};

foreach $file (<$members/*.dat>) {
open FIN,"<$file";
($fout,$name)=tempfile;
while ($str=<FIN>) {
print $fout $str unless $str=~m/^site(url|name)\s*=/;
}
close $fout;
close FIN;

rename $name,$file;
}
1;
 
Hi

So it is a CGI script. That makes the debugging abit complicated. The steps I would do :
[ul]
[li]Check if the user with who's permission the web server runs, is allowed to write those files.[/li]
[li]Add [tt]warningsToBrowser[/tt] :
Code:
[gray]# change this[/gray]
[b]use[/b] CGI[teal]::[/teal]Carp [b]qw[/b][teal]{[/teal]fatalsToBrowser [highlight]warningsToBrowser[/highlight][teal]}[/teal][teal];[/teal]

[gray]# ...[/gray]

[gray]# add this after the HTTP headers were sent[/gray]
[highlight][COLOR=darkgoldenrod]warningsToBrowser[/color][teal]([/teal][purple]1[/purple][teal]);[/teal][/highlight]
[/li]
[li]Check your web server's error log.[/li]
[li]Add some [tt]print[/tt] calls to see what gets executed :
Code:
[gray]# ...[/gray]

[highlight][b]print[/b] [green][i]"start dat editing <br>\n"[/i][/green][teal];[/teal][/highlight]
[b]use[/b] File[teal]::[/teal]Temp [b]qw[/b][teal]{[/teal]tempfile[teal]}[/teal][teal];[/teal]

[highlight][b]print[/b] [green][i]"directory is $members<br>\n"[/i][/green][teal];[/teal][/highlight]
[highlight][b]print[/b] [green][i]"files to process are "[/i][/green][teal],[/teal][green][i]<$members/*.dat>[/i][/green][teal],[/teal][green][i]"<br>\n"[/i][/green][teal];[/teal][/highlight]
[b]foreach[/b] [navy]$file[/navy] [teal]([/teal][green][i]<$members/*.dat>[/i][/green][teal])[/teal] [teal]{[/teal]
  [highlight][b]print[/b] [green][i]"start file $file<br>\n"[/i][/green][teal];[/teal][/highlight]
  [b]open[/b] FIN[teal],[/teal][green][i]"<$file"[/i][/green][teal];[/teal]
  [teal]([/teal][navy]$fout[/navy][teal],[/teal][navy]$name[/navy][teal])=[/teal]tempfile[teal];[/teal]
  [highlight][b]print[/b] [green][i]"write temp file $name<br>\n"[/i][/green][teal];[/teal][/highlight]
  [b]while[/b] [teal]([/teal][navy]$str[/navy][teal]=[/teal][green][i]<FIN>[/i][/green][teal])[/teal] [teal]{[/teal]
    [b]print[/b] [navy]$fout[/navy] [navy]$str[/navy] [b]unless[/b] [navy]$str[/navy][teal]=~[/teal][b]m[/b][fuchsia]/^site(url|name)\s*=/[/fuchsia][teal];[/teal]
  [teal]}[/teal]
  [b]close[/b] [navy]$fout[/navy][teal];[/teal]
  [b]close[/b] FIN[teal];[/teal]

  [highlight][b]print[/b] [green][i]"rename file $name to $file<br>\n"[/i][/green][teal];[/teal][/highlight]
  [b]rename[/b] [navy]$name[/navy][teal],[/teal][navy]$file[/navy][teal];[/teal]
  [highlight][b]print[/b] [green][i]"end file $file<br>\n"[/i][/green][teal];[/teal][/highlight]
[teal]}[/teal]
[highlight][b]print[/b] [green][i]"end dat editing<br>\n"[/i][/green][teal];[/teal][/highlight]
[purple]1[/purple][teal];[/teal]
[/li]
[/ul]


Feherke.
[link feherke.github.com/][/url]
 
It looks like it is functioning but it doesn't actually delete the lines siteurl and sitename.


start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat./Members/5.dat
start file ./Members/1.dat
write temp file /tmp/WFTaSsK1ga
rename file /tmp/WFTaSsK1ga to ./Members/1.dat
end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/b5adpSuXC5
rename file /tmp/b5adpSuXC5 to ./Members/2.dat
end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/TSvi8giLoH
rename file /tmp/TSvi8giLoH to ./Members/3.dat
end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/CBfa12yMke
rename file /tmp/CBfa12yMke to ./Members/4.dat
end file ./Members/4.dat
start file ./Members/5.dat
write temp file /tmp/cOaQsiwo82
rename file /tmp/cOaQsiwo82 to ./Members/5.dat
end file ./Members/5.dat
end dat editing
Content-type: text/html; charset=UTF-8
 
Hi

Seems my first point was right :
Feherke said:
[ul][li]Check if the user with who's permission the web server runs, is allowed to write those files.[/li][/ul]

To be sure, you can add one more [tt]print[/tt] :
Code:
[highlight][b]print[/b][/highlight] [b]rename[/b] [navy]$name[/navy][teal],[/teal][navy]$file[/navy][teal];[/teal]

It will output 1 on success and 0 on failure. Probably will be all 0s. This may be caused by more things, but most probably is the lack of write permission.

To correct this give write permission on Members/ directory. So from the command line first become superuser, then execute [tt]chmod a+x /path/to/Members/[/tt]. ( Of course, in place of [tt]a[/tt] ( all ) use [tt]g[/tt] ( group ) in case the web server's configuration sets the group to the same group as Members/ is owned by. )

If still not works, you will have to add it to the files themselves with [tt]chmod a+x /path/to/Members/*.dat[/tt]. But in this case you will have the same problem after creating/generating newer dat files. ( I suppose there will be more of them later. )

Feherke.
[link feherke.github.com/][/url]
 
Thanks, almost there.
It is now removing the line sitename but not the line siteurl

There are full write permissions on the folder.
 
Hi

Strange. Are you sure the sample in your opening post is correct ? I tested with that one and worked correctly. ( Given that siteurl is the last line, I double-checked both with and without trailing newline. )

Maybe there is a leading space in that line ? If yes, you can get ride of it with this :
Code:
[b]print[/b] [navy]$fout[/navy] [navy]$str[/navy] [b]unless[/b] [navy]$str[/navy][teal]=~[/teal][b]m[/b][fuchsia]/^[highlight]\s*[/highlight]site(url|name)\s*=/[/fuchsia][teal];[/teal]

Feherke.
[link feherke.github.com/][/url]
 
My mistake, it's not deleting any of the lines, it just reorderd them.

Here's what I now get:

start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat./Members/5.dat
start file ./Members/1.dat
write temp file /tmp/nKijKk93AQ
rename file /tmp/nKijKk93AQ to ./Members/1.dat
0end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/6ydGcqERS1
rename file /tmp/6ydGcqERS1 to ./Members/2.dat
0end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/Ck7RE8hTNL
rename file /tmp/Ck7RE8hTNL to ./Members/3.dat
0end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/rSsiIt7fF0
rename file /tmp/rSsiIt7fF0 to ./Members/4.dat
0end file ./Members/4.dat
start file ./Members/5.dat
write temp file /tmp/px78Uqg55x
rename file /tmp/px78Uqg55x to ./Members/5.dat
0end file ./Members/5.dat
end dat editing
 
Hi

Well, those 0s in the output are clearly showing that [tt]rename[/tt] is still failing.

There is another step to include to see the exact reason of the failure :
Code:
[b]print[/b] [green][i]"rename file $name to $file<br>\n"[/i][/green][teal];[/teal]
[b]print[/b] [b]rename[/b] [navy]$name[/navy][teal],[/teal][navy]$file[/navy][teal];[/teal]
[highlight][b]print[/b] [green][i]"rename error $!<br>\n"[/i][/green][teal];[/teal][/highlight]
[highlight][b]undef[/b] $[teal]!;[/teal][/highlight]
[b]print[/b] [green][i]"end file $file<br>\n"[/i][/green][teal];[/teal]


Feherke.
[link feherke.github.com/][/url]
 
start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat
start file ./Members/1.dat
write temp file /tmp/08yw9L8QjX
rename file /tmp/08yw9L8QjX to ./Members/1.dat
0rename error Invalid cross-device link
end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/DXfbMJ81WL
rename file /tmp/DXfbMJ81WL to ./Members/2.dat
0rename error Invalid cross-device link
end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/FKbrf0HXol
rename file /tmp/FKbrf0HXol to ./Members/3.dat
0rename error Invalid cross-device link
end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/iSihH4mK3P
rename file /tmp/iSihH4mK3P to ./Members/4.dat
0rename error Invalid cross-device link
end file ./Members/4.dat
end dat editing
 
Hi

Ah. Got it. Your Members/ and /tmp/ directories are on different partitions. Then something stronger than the basic [tt]rename[/tt] is needed :
Perl:
[b]use[/b] File[teal]::[/teal]Temp [b]qw[/b][teal]{[/teal]tempfile[teal]}[/teal][teal];[/teal]
[highlight][b]use[/b] File[teal]::[/teal]Copy[teal];[/teal][/highlight]

[b]foreach[/b] [navy]$file[/navy] [teal]([/teal][green][i]<*.dat>[/i][/green][teal])[/teal] [teal]{[/teal]
  [b]open[/b] FIN[teal],[/teal][green][i]"<$file"[/i][/green][teal];[/teal]
  [teal]([/teal][navy]$fout[/navy][teal],[/teal][navy]$name[/navy][teal])=[/teal]tempfile[teal];[/teal]
  [b]while[/b] [teal]([/teal][navy]$str[/navy][teal]=[/teal][green][i]<FIN>[/i][/green][teal])[/teal] [teal]{[/teal]
    [b]print[/b] [navy]$fout[/navy] [navy]$str[/navy] [b]unless[/b] [navy]$str[/navy][teal]=~[/teal][b]m[/b][fuchsia]/^site(url|name)\s*=/[/fuchsia][teal];[/teal]
  [teal]}[/teal]
  [b]close[/b] [navy]$fout[/navy][teal];[/teal]
  [b]close[/b] FIN[teal];[/teal]

  [highlight]move[/highlight] [navy]$name[/navy][teal],[/teal][navy]$file[/navy][teal];[/teal]
[teal]}[/teal]

Feherke.
[link feherke.github.com/][/url]
 
You are a genius Sir, thank you very much, all working like a charm.
 
One last question.
Is it possible to skip if the user is administrator?

This is the line in the .dat file

sn = |Administrator|


Cheers
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top