CNC machine text dump --> Database using VB

[Legions]

I am working on a client mini-app upgrade. They use this very specific CNC machine interface that kicks results per job out via a TEXT file. This text file gets VERY large by the end of the job.

The previous app that was built would just constantly grab the same text file location on the network and load it into a database (access97), then compare, and parse the changes out.

The problem is that the CNC has been upgraded and is handling larger jobs - the previous app can't handle the text file size load, as it gets very large after 4 - 8 hours of non-stop running.

So I am looking into solutions - I have run into a stumbling block that the Text Dump file has to continue to happen. The CNC machine appends this one text file non-stop for 4 - 8 hours.

Each line has a time/date entry and the dimensions / info. I have all the case statements in VB 6 to read, translate, and react to end user as it happens real time - the problem is the text file is SLOW as it gets super large.

Is there a way in VB6 to open the text file and just pull in the NEW line of code constantly - instead of grabbing the entire text file and loading the entire thing, and then pulling out the NEW line?

It has to be as efficient as possible.

Any help or direction is great.

- L

 

[gmmastros]

the problem is the text file is SLOW as it gets super large

Can you be more specific. How large does this file get? It could be that loading it a different way will solve your problem.

-George

Strong and bitter words indicate a weak cause. - Fortune cookie wisdom

 

[Legions]

I'm loading the Entire text file into MS-SQL server and/or Access 2002.

The program parses the text file about 44k - 1 meg size, removes unneeded information (50%), and then reads line by line of the raw data and saves only the lines that are important.

I want to have VB6 app, pull the textfile (fileobject), but keep it open?? because the CNC machine dumpes line by line every 5 - 10 seconds.

If you have any suggestion - greatly appreciate it.

- L

 

[strongm]

We're still waiting to find out how large this file gets.

 

[Legions]

44k Start
-
-
-
-
1.9 megs - about halfway through
-
-
-
-
3.4 - 4 megs - finished

Average jobs take 2 megs, some alot less 100 k, larger jobs take up to 5 megs. Old tool crashes at 350 - 500 k mark.

 

[TGandon]

Here is an initial suggestion using a timer control and reading blocks of data from the file.

Edit the constants to suit.

Option Explicit

Const CNC_TEXT_DUMP As String = "C:\MyFolder\MySubFolder\CNCDUMP.TXT"
Const MAX_BUFFER_SIZE As Long = 16384
Const MIN_SIZE_INCREASE As Long = 256

Dim mlngFileSize As Long
Dim mlngInputPtr As Long

Private Sub Form_Load()

mlngInputPtr = 1

End Sub

Private Sub Timer1_Timer()

Dim lngBufferSize As Long, intIndex As Integer
Dim strBuffer As String, strData() As String

If FileLen(CNC_TEXT_DUMP) > mlngInputPtr + MIN_SIZE_INCREASE Then
   mlngFileSize = FileLen(CNC_TEXT_DUMP)
   If mlngFileSize - mlngInputPtr > MAX_BUFFER_SIZE Then
      lngBufferSize = MAX_BUFFER_SIZE
   Else
      lngBufferSize = mlngFileSize - mlngInputPtr + 1
   End If
   
   strBuffer = String(lngBufferSize, vbNullChar)

   Open CNC_TEXT_DUMP For Binary Access Read Shared As #1
   Get #1, mlngInputPtr, strBuffer
   Close 1
   
   ' Restrict strBuffer to contain only whole lines of data
   If InStr(strBuffer, vbCr) > 0 Then
      strBuffer = Left(strBuffer, Len(strBuffer) - InStr(StrReverse(strBuffer), vbCr))
      mlngInputPtr = mlngInputPtr + Len(strBuffer) + 2
   Else
      mlngInputPtr = mlngInputPtr + lngBufferSize
   End If
   
   strData = Split(strBuffer, vbCrLf)
   For intIndex = 0 To UBound(strData)
   
      ' Process each row of strData
      
   Next
   
End If

End Sub

Trevor