Reading a file sequentially 1

[philclare]

Hi,

I have a file consisting of a number of lines of text, I want to read the file in order and execute some code based on the content of each line.

My problem is that I only want to access each line once and because of the size of the file, my script times out, so only gets so far down the file. If I then run the script again it starts from the beginning of the file rather than from where it left off.

My idea is to delete each line of the file after it has been used, however I can't find out how to do this or if this is the best way...

Unfortunately I don't have administrative access to the server so I can't simply extend the script timeout (and wouldn't want to anyway)

Any ideas anybody?

Thanks

Hope this helps,

Phil Clare

http://www.trinipoint.co.uk

 

[sleipnir214]

I have a few ideas.

If available, dump the file into a database server, then procuess each record, deleting as you go.

Maintain a line counter in a separate file. As you finish processing a line, increment the counter in the file.

Figure out how many records you can reliably process during one run of your script. Then have the script read out that number of lines into memory, write the remainder to a new file, delete the old file, then rename the old file to the new file name. The process those lines it's read into memory.


Keep in mind the problems you may face with web scripts -- they can be run multiple times simultaneously. If this could be a problem for you, make sure you use file locking to keep multiple instantiations of the script from interfering with one another's reading of the batch file.

Want the best answers? Ask the best questions:

http://www.catb.org/~esr/faqs/smart-questions.html

TANSTAAFL!!

 

[philclare]

Well, the file is a list of SQL statements but I can't do an import in phpmyadmin or anything because the upload times out due to my ISP, the script is only for me to use and will be in a password protected folder so I don't have to worry about other people using it.

I think the record counter in a second file is the best idea, i think I'll do that

Happy New Year

Hope this helps,

Phil Clare

http://www.trinipoint.co.uk

 

[jstreich]

Alternitivly, read the who file into an array, and after each line executes, recall the page passing the remaining array, when there is nothing in the array, stop. This way your not eating the file each time by redeleting it's contents, and you don't have to go to a database. The other good thing is output between commands for errors or an option to stop in the middle...