Split Text File Into Smaller Text Files

[air1access]

I have searched and searched... I cannot find anything that I can make work for me..
I have a huge text file (about 4mil lines/records). I like to break it up into smaller text files, in the same location.
Something like read the first 200,000 lines of the original, write that to a new text file, then pick up on line 200,001 of the original, read the next 200,000, then write that to another new text file - so on & so on until its done.

Any examples out there..? Any suggestions..?
Thanks in advance..!!!

 

[Andrzejek]

Code not tested:

Dim aryTemp() As String
Dim strInputData As String
Dim lngC As Long
Dim iCounter As Integer

Open "C:\TheBigTextFile.txt" For Input As #1
strInputData = Input$(LOF(1), 1)
[green]'Close #1[/green]
aryTemp = Split(strInputData, vbCrLf)

For lngC = LBound(aryTemp) To UBound(aryTemp)
    If lngC Mod 200000 = 0 Then
        iCounter = iCounter + 1
        Close #1
        Open "C:\SmallTextFile" & iCounter & ".txt" For Output As #1
    End If
    Print #1, aryTemp(lngC)
Next lngC
Close #1

I am sure there is a better way...

Have fun.

---- Andy

 

[randy700]

How about a series of queries?

qryFirst: SELECT TOP 200000 FROM BIG_FILE
qrySecond: SELECT TOP 200000 FROM BIG_FILE WHERE SOME_FIELD NOT IN
(SELECT TOP 200000 FROM BIG_FILE)
qryThird: SELECT TOP 200000 FROM BIG_FILE WHERE SOME_FILED NOT IN
(SELECT TOP 400000 FROM BIG_FILE)
etc.



Randy

 

[randy700]

Sorry, meant to click PREVIEW - clicked SUBMIT by mistake.

How about a series of queries?

qryFirst:   
SELECT TOP 200000 FROM BIG_FILE

qrySecond: 
SELECT TOP 200000 FROM BIG_FILE WHERE SOME_FIELD NOT IN 
(SELECT TOP 200000 FROM BIG_FILE)

qryThird:  
SELECT TOP 200000 FROM BIG_FILE WHERE SOME_FIELD NOT IN
(SELECT TOP 400000 FROM BIG_FILE)

etc.

Do this in VBA and you can substitute variables for the numbers.



Randy

 

[MakeItSo]

How about using Gsplit?
==>

http://www.gdgsoft.com/gsplit/

;-)

“Knowledge is power. Information is liberating. Education is the premise of progress, in every society, in every family.” (Kofi Annan)
Oppose SOPA, PIPA, ACTA; measures to curb freedom of information under whatever name whatsoever.

 

[MakeItSo]

Did you get it to work??

“Knowledge is power. Information is liberating. Education is the premise of progress, in every society, in every family.” (Kofi Annan)
Oppose SOPA, PIPA, ACTA; measures to curb freedom of information under whatever name whatsoever.

 

[air1access]

Yes - but it takes waaaaaaaay too long... I guess this file is just too big to split up. Its over 3million lines.. Literally..

 

[sxschech]

I've been using a program called xplorer2 for some time as a file management tool, it has many other capabilities such as merging files. It is also great for filtering files and has dual panes so you can move or copy files between folders and drives. I haven't had a need to split files, so hadn't looked into it, I figured if it could merge, it probably could split and turns out that this program can also split files. It is free for home or academic use and $30 for the professional. There is a free trial period on that, so you could test it out. Looks like you can specify the size of the split files.

http://www.zabkat.com/x2lite.htm

The website also has a video demo/tutorial on splitting.

http://www.zabkat.com/blog/26Jul09-split-and-join-files.htm

 

[strongm]

>Yes - but it takes waaaaaaaay too long

Which solution were you trying?