Line read a file in reverse order

[tilmant]

I am using the following script and its working quite well reading the files and parsing out those that have a "04/ in the line. The problem is is that some of my files have hundreds of thousands of lines and the lines that contain "04/ are at the end. Is it possible to quickly jump the pointer to the first line that contains "04/ and then start the read line or can I reverse the read line command so that it starts at the bottom (then have it in the correct order)?

----------------------------------------------------------

Dim objFSO, objFolder, colFiles, File, strDirName, objFile, objFile2, strOtherNewText, strNewText

Const ForReading = 1
Const ForWriting = 2
Const ForAppending = 8

strDirName = "temp"

Set objFSO = CreateObject("Scripting.FileSystemObject")
Set objFolder = objFSO.GetFolder(strDirName)
Set colFiles = objFolder.Files

For Each File in colFiles
Set objTextFile = objFSO.OpenTextFile(File.Path, ForReading)
Set objTextFileheader = objFSO.OpenTextFile("header.txt", ForReading)
fileArray = Split(file.Name,".")
newFileName = fileArray(0)
strNewText = objTextFileheader.ReadLine


Do Until objTextFile.AtEndOfStream
strLine = objTextFile.ReadLine
intFailure = InStr(strLine, chr(34) & "04/")
If intFailure > 0 Then
strNewText = strNewText & strLine & vbCrLf
Else
strOtherNewText = strOtherNewText & strLine & vbCrLf
End If
Loop

objTextFile.Close
objTextFileheader.Close
Set objTextFile = objFSO.OpenTextFile(newFileName&"-0407.txt" , ForWriting, True)
objTextFile.Write(strNewText)
Set objTextFile = objFSO.OpenTextFile(newFileName&"-0307.txt" , ForWriting, True)
objTextFile.Write(strOtherNewText)
objTextFile.Close
strOtherNewText = ""
strNewText = ""

Next

---------------------------------------------------------

 

[dm4ever]

So you're just looking for the lines in your file that contain the value of "04/"? If so, you may consider using Regular Expressions.

--------------------------------------------------------------------------------
dm4ever
My philosophy: K.I.S.S - Keep It Simple Stupid

 

[tilmant]

I'm not familiar with using regular expressions. I assumed that I would need to search for "04/ and then export just that line to a new file and delete it from the existing file. Can you point me to the example of how to do that?

 

[dm4ever]

Just so I have it straight...
1. look in a folder with text files
2. open the text file an look for lines with "04/"
3. write those lines to another file (will it be the same file for all those files read?)


--------------------------------------------------------------------------------
dm4ever
My philosophy: K.I.S.S - Keep It Simple Stupid

 

[dm4ever]

This is a simple example. It will look in your directory, open the files, search for "04/", and then write those lines to a text file

Option Explicit
'On Error Resume Next

Const ForReading = 1
Const ForWriting = 2
Const ForAppending = 8

Dim strDirName : strDirName = "C:\Temp\test"

Dim objFSO : Set objFSO = CreateObject("Scripting.FileSystemObject")
Dim objFolder : Set objFolder = objFSO.GetFolder(strDirName)
Dim objFile2 : Set objFile2 = objFSO.OpenTextFile("c:\temp\output.txt", ForWriting, True)

Dim RegEx : Set RegEx = New RegExp
RegEx.Pattern = ".*04\/.*"
RegEx.Global = True
RegEx.IgnoreCase = True

Dim file, objFile, colMatches, objMatch
For Each file In objFolder.Files
	Set objFile = objFSO.OpenTextFile(file.Path)
	Set colMatches = RegEx.Execute(objFile.ReadAll)
	objFile.Close
	For Each objMatch In colMatches
	   objFile2.WriteLine objMatch.Value
	Next
Next
objFile2.Close

--------------------------------------------------------------------------------
dm4ever
My philosophy: K.I.S.S - Keep It Simple Stupid

 

[tilmant]

Thanks dm4ever, no it will not be going to the same file so I will need to create a new file for each object. Based on your wonderful example I can make those changes easily. Now how can I delete the found lines from the original file?

 

[jfdabiri]

you can also use sql query to read and examine a text file. you will get a recordset of all the lines in the file. you can write some or all of the lines to another file, like dm4 instructed you. but you cannot delete lines from the original file. you would get an ISAM error on a linked table.

 

[dm4ever]

You might play with this to write back to the original files the content without those lines.

Option Explicit
'On Error Resume Next

Const ForReading = 1
Const ForWriting = 2
Const ForAppending = 8

Dim strDirName : strDirName = "C:\Temp\test"

Dim objFSO : Set objFSO = CreateObject("Scripting.FileSystemObject")
Dim objFolder : Set objFolder = objFSO.GetFolder(strDirName)
Dim objFile2 : Set objFile2 = objFSO.OpenTextFile("c:\temp\output.txt", ForWriting, True)

Dim RegEx : Set RegEx = New RegExp
RegEx.Global = True
RegEx.IgnoreCase = True

Dim file, objFile, colMatches, objMatch, strFileContent
For Each file In objFolder.Files
	Set objFile = objFSO.OpenTextFile(file.Path)
	strFileContent = objFile.ReadAll
	objFile.Close
	
	RegEx.Pattern = ".*04\/.*"
	Set colMatches = RegEx.Execute(strFileContent)
	For Each objMatch In colMatches
	   objFile2.WriteLine objMatch.Value
	Next
	
	RegEx.Pattern = ".*04\/.*\s?"
	strFileContent = RegEx.Replace(strFileContent, "")
	Set objFile = objFSO.OpenTextFile(file.Path, ForWriting, True)
	objFile.Write strFileContent
	objFile.Close
Next
objFile2.Close

--------------------------------------------------------------------------------
dm4ever
My philosophy: K.I.S.S - Keep It Simple Stupid

 

[kmcferrin]

Wouldn't that get you back to where he started though, in that you'd have to iterate through every line of the file?

 

[dm4ever]

However you look at it, you will need to read the entire content of the file to look for what you want to remove/keep what you want. Using a regular expression with large amount of data is sometimes faster.

Look at the post here to count the number of lines in a 1 Mil line text file:

http://www.visualbasicscript.com/m_45417/tm.htm

15 sec breaking it into an array
3 sec using Regular expression

This post

http://www.visualbasicscript.com/m_44624/mpage_1/key_massive/tm.htm#44652

the method posted was going on to 24hrs
i think using a regular expression on a 3Mil file took about 30sec


--------------------------------------------------------------------------------
dm4ever
My philosophy: K.I.S.S - Keep It Simple Stupid

 

[tsuji]

to: OP
If you want to read the file backward, you can do it with adodb set up on jet oledb provider. This may help you work through it.

http://tek-tips.com/viewthread.cfm?qid=1102598