Newbie Question 1

[martinjburgess]

Hi,

I have a folder with a list of files - 1.dat, 2.dat, 3.dat etc
The file structure is:

rndsid = |11026|1342296563|
lastactive = |1342296563|
lastvisit = |1340862364|
sitename = |Free PC Help|
email = |admin@mysite.com|
pmpopup = |1|
lastpost = |1340736611|
password = |******|
posts = |2|
registered = |1339327069|
dateformat = |0|
timezone = |0|
timeformat = |0|
sn = |Administrator|
siteurl = |

http://www.pcfixzone.co.uk|

What I need to do is read the content of each .dat file and remove the entries for siteurl and sitename if they exist.

Is this possible?

Many thanks

 

[feherke]

Hi

remove the entries for siteurl and sitename if they exist.

You mean, remove the entire such line ?

This does in-place modification, so better make backups before trying it :

perl -pi -e '[b]undef[/b][navy]$_[/navy] [b]if[/b][green][i]/^site(url|name)\s*=/[/i][/green]' *.dat

Feherke.
[link feherke.github.com/]

http://feherke.github.com/

[/url]

 

[martinjburgess]

Thanks for the reply but that just gives a 500 error

What I need is a small script that when run will:

open each .dat file in the Members folder
Read each line
If siteurl or sitename exists either delete the line or set it to ''

 

[feherke]

Hi

Thanks for the reply but that just gives a 500 error

What is "a 500 error" ? Show us how exactly you tried it and what exactly you got.


Feherke.
[link feherke.github.com/]

http://feherke.github.com/

[/url]

 

[martinjburgess]

I pasted it at the top of another script and when run it just gives a HTTP 500 error.

I think I will need something more on the lines of:

open(FILE,"$members/$_.dat");
while(<FILE>) {
code
code
code
close(FILE);
}

or something along those lines.

Sorry for being so vague.

 

[feherke]

Hi

I pasted it at the top of another script

[red]vvvvvvvvvvv--- execute it from the command line[/red]​

perl -pi -e '[b]undef[/b][navy]$_[/navy] [b]if[/b][green][i]/^site(url|name)\s*=/[/i][/green]' *.dat

just gives a HTTP 500 error.

How gets HTTP involved in all this ?

Anyway.

[b]use[/b] File[teal]::[/teal]Temp [b]qw[/b][teal]{[/teal]tempfile[teal]}[/teal][teal];[/teal]

[b]foreach[/b] [navy]$file[/navy] [teal]([/teal][green][i]<*.dat>[/i][/green][teal])[/teal] [teal]{[/teal]
  [b]open[/b] FIN[teal],[/teal][green][i]"<$file"[/i][/green][teal];[/teal]
  [teal]([/teal][navy]$fout[/navy][teal],[/teal][navy]$name[/navy][teal])=[/teal]tempfile[teal];[/teal]
  [b]while[/b] [teal]([/teal][navy]$str[/navy][teal]=[/teal][green][i]<FIN>[/i][/green][teal])[/teal] [teal]{[/teal]
    [b]print[/b] [navy]$fout[/navy] [navy]$str[/navy] [b]unless[/b] [navy]$str[/navy][teal]=~[/teal][b]m[/b][fuchsia]/^site(url|name)\s*=/[/fuchsia][teal];[/teal]
  [teal]}[/teal]
  [b]close[/b] [navy]$fout[/navy][teal];[/teal]
  [b]close[/b] FIN[teal];[/teal]

  [b]rename[/b] [navy]$name[/navy][teal],[/teal][navy]$file[/navy][teal];[/teal]
[teal]}[/teal]

Next time please specify all requirements at the beginning.

Feherke.
[link feherke.github.com/]

http://feherke.github.com/

[/url]

 

[martinjburgess]

Sorry but no joy.

I did change this:

foreach $file (<*.dat>) {

to this:

foreach $file (<$members/*.dat>) {

as this is the folder the .dat files are in.

 

[feherke]

Hi

Show us how exactly you tried it and what exactly you got.

Feherke.
[link feherke.github.com/]

http://feherke.github.com/

[/url]

 

[martinjburgess]

Thanks for your patience.
This is the code and nothing happens:

#!/usr/bin/perl

use CGI::Carp fatalsToBrowser;

# Global information:
$theblahver = 16;
$version = $versioncr = '10.3.6'; # Said Version; Copyright version
$blanktarget = " target='_new'";

# Uncomment this for better time precision
use Time::HiRes qw(time);

# Filename information
$scriptname = $scriptname || 'Blah.pl'; # Change name of Blah.pl
$modrewrite = $modrewrite || '?'; # Setting, mod_rewrite: on = '' | off = '?'

use Fcntl ':flock';

# Default language
$languagep = "English";
$languages = "./Languages";

require('Settings.pl');

require("$code/QuickCore.pl");
UFS();
CheckCookies();
GetThemes();

$language = "$languages/$languagep";
require("$language.lng");
require("$code/Routines.pl");
require("$code/Load.pl");


# Remove the theme variable for guests/search engines
redirect() if($URL{'theme'} && $username eq 'Guest');

# Load basic features we can use later
CreateGroups();
BoardCheck();
ClickLog();
AL();

use File::Temp qw{tempfile};

foreach $file (<$members/*.dat>) {
open FIN,"<$file";
($fout,$name)=tempfile;
while ($str=<FIN>) {
print $fout $str unless $str=~m/^site(url|name)\s*=/;
}
close $fout;
close FIN;

rename $name,$file;
}
1;

 

[feherke]

Hi

So it is a CGI script. That makes the debugging abit complicated. The steps I would do :
[ul]
[li]Check if the user with who's permission the web server runs, is allowed to write those files.[/li]
[li]Add [tt]warningsToBrowser[/tt] :

[gray]# change this[/gray]
[b]use[/b] CGI[teal]::[/teal]Carp [b]qw[/b][teal]{[/teal]fatalsToBrowser [highlight]warningsToBrowser[/highlight][teal]}[/teal][teal];[/teal]

[gray]# ...[/gray]

[gray]# add this after the HTTP headers were sent[/gray]
[highlight][COLOR=darkgoldenrod]warningsToBrowser[/color][teal]([/teal][purple]1[/purple][teal]);[/teal][/highlight]

[/li]
[li]Check your web server's error log.[/li]
[li]Add some [tt]print[/tt] calls to see what gets executed :

[gray]# ...[/gray]

[highlight][b]print[/b] [green][i]"start dat editing <br>\n"[/i][/green][teal];[/teal][/highlight]
[b]use[/b] File[teal]::[/teal]Temp [b]qw[/b][teal]{[/teal]tempfile[teal]}[/teal][teal];[/teal]

[highlight][b]print[/b] [green][i]"directory is $members<br>\n"[/i][/green][teal];[/teal][/highlight]
[highlight][b]print[/b] [green][i]"files to process are "[/i][/green][teal],[/teal][green][i]<$members/*.dat>[/i][/green][teal],[/teal][green][i]"<br>\n"[/i][/green][teal];[/teal][/highlight]
[b]foreach[/b] [navy]$file[/navy] [teal]([/teal][green][i]<$members/*.dat>[/i][/green][teal])[/teal] [teal]{[/teal]
  [highlight][b]print[/b] [green][i]"start file $file<br>\n"[/i][/green][teal];[/teal][/highlight]
  [b]open[/b] FIN[teal],[/teal][green][i]"<$file"[/i][/green][teal];[/teal]
  [teal]([/teal][navy]$fout[/navy][teal],[/teal][navy]$name[/navy][teal])=[/teal]tempfile[teal];[/teal]
  [highlight][b]print[/b] [green][i]"write temp file $name<br>\n"[/i][/green][teal];[/teal][/highlight]
  [b]while[/b] [teal]([/teal][navy]$str[/navy][teal]=[/teal][green][i]<FIN>[/i][/green][teal])[/teal] [teal]{[/teal]
    [b]print[/b] [navy]$fout[/navy] [navy]$str[/navy] [b]unless[/b] [navy]$str[/navy][teal]=~[/teal][b]m[/b][fuchsia]/^site(url|name)\s*=/[/fuchsia][teal];[/teal]
  [teal]}[/teal]
  [b]close[/b] [navy]$fout[/navy][teal];[/teal]
  [b]close[/b] FIN[teal];[/teal]

  [highlight][b]print[/b] [green][i]"rename file $name to $file<br>\n"[/i][/green][teal];[/teal][/highlight]
  [b]rename[/b] [navy]$name[/navy][teal],[/teal][navy]$file[/navy][teal];[/teal]
  [highlight][b]print[/b] [green][i]"end file $file<br>\n"[/i][/green][teal];[/teal][/highlight]
[teal]}[/teal]
[highlight][b]print[/b] [green][i]"end dat editing<br>\n"[/i][/green][teal];[/teal][/highlight]
[purple]1[/purple][teal];[/teal]

[/li]
[/ul]


Feherke.
[link feherke.github.com/]

http://feherke.github.com/

[/url]

 

[martinjburgess]

It looks like it is functioning but it doesn't actually delete the lines siteurl and sitename.


start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat./Members/5.dat
start file ./Members/1.dat
write temp file /tmp/WFTaSsK1ga
rename file /tmp/WFTaSsK1ga to ./Members/1.dat
end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/b5adpSuXC5
rename file /tmp/b5adpSuXC5 to ./Members/2.dat
end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/TSvi8giLoH
rename file /tmp/TSvi8giLoH to ./Members/3.dat
end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/CBfa12yMke
rename file /tmp/CBfa12yMke to ./Members/4.dat
end file ./Members/4.dat
start file ./Members/5.dat
write temp file /tmp/cOaQsiwo82
rename file /tmp/cOaQsiwo82 to ./Members/5.dat
end file ./Members/5.dat
end dat editing
Content-type: text/html; charset=UTF-8

 

[feherke]

Hi

Seems my first point was right :

[ul][li]Check if the user with who's permission the web server runs, is allowed to write those files.[/li][/ul]

To be sure, you can add one more [tt]print[/tt] :

[highlight][b]print[/b][/highlight] [b]rename[/b] [navy]$name[/navy][teal],[/teal][navy]$file[/navy][teal];[/teal]

It will output 1 on success and 0 on failure. Probably will be all 0s. This may be caused by more things, but most probably is the lack of write permission.

To correct this give write permission on Members/ directory. So from the command line first become superuser, then execute [tt]chmod a+x /path/to/Members/[/tt]. ( Of course, in place of [tt]a[/tt] ( all ) use [tt]g[/tt] ( group ) in case the web server's configuration sets the group to the same group as Members/ is owned by. )

If still not works, you will have to add it to the files themselves with [tt]chmod a+x /path/to/Members/*.dat[/tt]. But in this case you will have the same problem after creating/generating newer dat files. ( I suppose there will be more of them later. )

Feherke.
[link feherke.github.com/]

http://feherke.github.com/

[/url]

 

[martinjburgess]

Thanks, almost there.
It is now removing the line sitename but not the line siteurl

There are full write permissions on the folder.

 

[feherke]

Hi

Strange. Are you sure the sample in your opening post is correct ? I tested with that one and worked correctly. ( Given that siteurl is the last line, I double-checked both with and without trailing newline. )

Maybe there is a leading space in that line ? If yes, you can get ride of it with this :

[b]print[/b] [navy]$fout[/navy] [navy]$str[/navy] [b]unless[/b] [navy]$str[/navy][teal]=~[/teal][b]m[/b][fuchsia]/^[highlight]\s*[/highlight]site(url|name)\s*=/[/fuchsia][teal];[/teal]

Feherke.
[link feherke.github.com/]

http://feherke.github.com/

[/url]

 

[martinjburgess]

My mistake, it's not deleting any of the lines, it just reorderd them.

Here's what I now get:

start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat./Members/5.dat
start file ./Members/1.dat
write temp file /tmp/nKijKk93AQ
rename file /tmp/nKijKk93AQ to ./Members/1.dat
0end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/6ydGcqERS1
rename file /tmp/6ydGcqERS1 to ./Members/2.dat
0end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/Ck7RE8hTNL
rename file /tmp/Ck7RE8hTNL to ./Members/3.dat
0end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/rSsiIt7fF0
rename file /tmp/rSsiIt7fF0 to ./Members/4.dat
0end file ./Members/4.dat
start file ./Members/5.dat
write temp file /tmp/px78Uqg55x
rename file /tmp/px78Uqg55x to ./Members/5.dat
0end file ./Members/5.dat
end dat editing

 

[feherke]

Hi

Well, those 0s in the output are clearly showing that [tt]rename[/tt] is still failing.

There is another step to include to see the exact reason of the failure :

[b]print[/b] [green][i]"rename file $name to $file<br>\n"[/i][/green][teal];[/teal]
[b]print[/b] [b]rename[/b] [navy]$name[/navy][teal],[/teal][navy]$file[/navy][teal];[/teal]
[highlight][b]print[/b] [green][i]"rename error $!<br>\n"[/i][/green][teal];[/teal][/highlight]
[highlight][b]undef[/b] $[teal]!;[/teal][/highlight]
[b]print[/b] [green][i]"end file $file<br>\n"[/i][/green][teal];[/teal]

Feherke.
[link feherke.github.com/]

http://feherke.github.com/

[/url]

 

[martinjburgess]

start dat editing
directory is ./Members
files to process are ./Members/1.dat./Members/2.dat./Members/3.dat./Members/4.dat
start file ./Members/1.dat
write temp file /tmp/08yw9L8QjX
rename file /tmp/08yw9L8QjX to ./Members/1.dat
0rename error Invalid cross-device link
end file ./Members/1.dat
start file ./Members/2.dat
write temp file /tmp/DXfbMJ81WL
rename file /tmp/DXfbMJ81WL to ./Members/2.dat
0rename error Invalid cross-device link
end file ./Members/2.dat
start file ./Members/3.dat
write temp file /tmp/FKbrf0HXol
rename file /tmp/FKbrf0HXol to ./Members/3.dat
0rename error Invalid cross-device link
end file ./Members/3.dat
start file ./Members/4.dat
write temp file /tmp/iSihH4mK3P
rename file /tmp/iSihH4mK3P to ./Members/4.dat
0rename error Invalid cross-device link
end file ./Members/4.dat
end dat editing

 

[feherke]

Hi

Ah. Got it. Your Members/ and /tmp/ directories are on different partitions. Then something stronger than the basic [tt]rename[/tt] is needed :

[b]use[/b] File[teal]::[/teal]Temp [b]qw[/b][teal]{[/teal]tempfile[teal]}[/teal][teal];[/teal]
[highlight][b]use[/b] File[teal]::[/teal]Copy[teal];[/teal][/highlight]

[b]foreach[/b] [navy]$file[/navy] [teal]([/teal][green][i]<*.dat>[/i][/green][teal])[/teal] [teal]{[/teal]
  [b]open[/b] FIN[teal],[/teal][green][i]"<$file"[/i][/green][teal];[/teal]
  [teal]([/teal][navy]$fout[/navy][teal],[/teal][navy]$name[/navy][teal])=[/teal]tempfile[teal];[/teal]
  [b]while[/b] [teal]([/teal][navy]$str[/navy][teal]=[/teal][green][i]<FIN>[/i][/green][teal])[/teal] [teal]{[/teal]
    [b]print[/b] [navy]$fout[/navy] [navy]$str[/navy] [b]unless[/b] [navy]$str[/navy][teal]=~[/teal][b]m[/b][fuchsia]/^site(url|name)\s*=/[/fuchsia][teal];[/teal]
  [teal]}[/teal]
  [b]close[/b] [navy]$fout[/navy][teal];[/teal]
  [b]close[/b] FIN[teal];[/teal]

  [highlight]move[/highlight] [navy]$name[/navy][teal],[/teal][navy]$file[/navy][teal];[/teal]
[teal]}[/teal]

Feherke.
[link feherke.github.com/]

http://feherke.github.com/

[/url]

 

[martinjburgess]

You are a genius Sir, thank you very much, all working like a charm.

 

[martinjburgess]

One last question.
Is it possible to skip if the user is administrator?

This is the line in the .dat file

sn = |Administrator|


Cheers