Debugging VFP "0x00000000" Memory

[stanlyn]

Hi,

How can I debug this?

This message is coming from VFP when I turn off the Just-In-Time" debugger. When the JIT debugger is enabled, it throws an error, but I cannot use it to see anything about the syncdri.exe vfp executable.

The error in the dialog from vfp is...

Caption = "syncdri.exe - Application Error"
The instruction at "0x00000000" referenced memory at "0x00000000". The memory could not be "read".
Click OK to terminate the program
Click on Cancel to debug the program...

When running it with one set of local VFP data, it runs without errors, while another set is generating this error. It may also process several thousand or less than a hundred records before erroring out. I've inspected the records that was being processed when the error occurs, and see nothing, as the VFP data is ok.

It is updating a SQL Server 2008r2 backend with the VFP data via a native sql odbc connection. Both data sets uses the same connection string to the back end database and tables. Being said, I believe all the connection string and backend database stuff is ok, otherwise I'd be having issues with all sets of VFP data, and thats not the case.

Any ideas on how I might troubleshoot this?

Thanks, Stanley

 

[stanlyn]

OK... more info...

Here is a screen capture when the JIT debugger is enabled. It's message is:

An unhandled win32 execption occurred in syncdri.exe [10588].

It only lists the VS2008 debugger as a possible debugger...
Is it possible to include the VFP debugger into this list here?

Thanks, Stanley

 

[Olaf Doschke]

No, the foxpro debugger only debugs foxpro code running within the IDE, not in an EXE.

This error comes from outside VFP, it may come from the ODBC driver, in VFP the only non foxpro errors you'd get are c0000005 and some others from the vfp runtime, which itself uses the C++ runtime, VFP is developed in C++. Unless you don't use SYS(2700) to directly write to or read from memory foxpros memory allocation makes it quite impossible to make a memory access violation, which would lead to such error.

To me this rather looks like an ODBC problem or even hardware RAM failures, eg using non ECC Ram or wrong Ram settings, latency timings.

In the first place, the newest foxpro odbc driver was done for VFP6, that in itself would only be a problem, if you use field types like varchar, etc, introduced after VFP6. But it also is old in itself and you may see first issues of it running unstable on newer OSes.

In the log run, try to migrat to SSIS using OLEDB Provider Datasource to connect to DBC/DBFs.

Bye, Olaf.

 

[stanlyn]

Are you saying that using sys(2700) which is "Enables XP Themes" could lead to this problem? Why would themes have anything to do with it?

If it were an odbc or memory issue, then wonder why one of my data sets processed 325,000 records that took about 20 hours to do never had a single error, and now this 2nd data set can not do more than 1000 before crashing?

Thanks, Stanley

 

[Olaf Doschke]

Sorry, wrong number. I meant SYS(2600), which enables writing to memory addresses you get by some Windows API calls.

Themes have nothing to do with this.

Bye, Olaf.

 

[stanlyn]

Update...

When looking at the differences between the working and failing data sets, I noticed that the failing one has more than double the records to process... 325,000 compared to 760,000. So I added some code that limited the results to process down to 200-300K range and ran it. Its 3:14PM now and its been running nonstop (10+ hours) since 5:10AM this morning, and all without a hitch...

For every record, it...
1. creates a new vfp record in an archive vfp table that contains all the vfp source table's data including opening a filesystem image file and inserting the binary data of the image file into a vfp blob field. This serves as a permanent backup archive that an import can read.

2. it searches the Sql Server table for a matching entry and if found it updates the sql server table, again inserting the original .sla image data into a sql varbinary(max) field. If not found, it inserts a new record with all the data,

3. Next it converts the original .sla bitmap image to a .tif file and only saves that to the sql server image_tif field which is varbinary(max),

4. And of-course, full logging to a filesystem file and an editbox control.

So, I don't know if its the extra records or was it processing a bunch of bad images, (.sla, or .tif files) I'll keep processing the records in 250k chunks and if the problem reappears, then it would have to be the image processing...

I'll start using the sys(2600) and see if that makes a difference.

Thanks, Stanley

 

[stanlyn]

Olaf,

I looked at the sys(2600) function and would'nt know where to start in its implementation... It did say it was "For Advanced Programmers Only"..., which obviously I'm not there yet...

Thanks, Stanley

 

[Olaf Doschke]

You still didn't get it. I just said SYS(2600) could contribute to memory access errors you experience. This rather means NOT to use it instead of using it.

SYS(2600) is a function letting you access memory, also memory not belonging to your process, so it coudl lead to errors. Forget about it, if you didn't needed it yet, you dan't need to start to use it.

From your descrition it turns out lage chunks of data are leading to the memory errors. This leads me back to the suspicion the RAM hardware is flaky. Maybe too large chuns of data also lead to swaping, needing more than physical RAM available, which obviously is to avaoid, also from the point of view of performance.

Bye, Olaf.

 

[stanlyn]

Good, I'll forget it, as I've never used it and from the looks of it, its more complicated than what I have time for...

Still waiting for the sql server to finish a table restructure that been going on now for 12+ hours... Will resume processing as soon as the restructure is done.

I'm going to let it finish using the smaller selections limiting them to about 250,000 records. The image files are 10-80kb in size and I hope the problem is there as I'd rather be dealing with a bad image issues instead of vfp not processing more than 250000 records at a time which would be memory related... If it finishes them all then, 1. we do not have bad image issues, and 2. swap out the memory... Do you see it this way?

Thanks, Stanley

 

[Olaf Doschke]

Bag image issues? Just because your moving images from disc to blob to server you think you get memory errors? This is just moving the file bytes, could also be anything else and not causin memory errors.

If an image is flaky and causes trouble, then only when displaying, when being "interpreted" and decompressed, false bytes could lead to false processing, which indeed also can lead to access of wrong memory adresses, but not when you transport an image file, this can not at all be related to bad image data.

I step 2 above I wonder if you compare records including the image data. Doing this record wise also takes a huge amount of time and overhead. Why don't you SET COVERAGE and monitor what steps need what time, so you know what to optimise?

Bye, Olaf.

 

[stanlyn]

No Olaf,

<< If an image is flaky and causes trouble, then only when displaying, when being "interpreted" and decompressed, false bytes could lead to false processing, which indeed also can lead to access of wrong memory adresses, but not when you transport an image file, this can not at all be related to bad image data.

Part of getting this image from disk to a sql fields involves:
1. placing a copy of the original image into an image control, then
2. converting the image control's image into a .tif file, then
3. copy the converted image's binary into the image_tif sql field, and
4. copies the original disk image's binary into the image_sla sql field.

If the image is bad, which I do NOT believe them to be, (because I can view the one in question via Winexplorer), then there are many places in this process for it to blow up...

Am I wrong?

Thanks, Stanley

 

[Olaf Doschke]

OK, then you're right, but how shoul I know you load the image into a image control to convert it to TIF?

You said you convert it to TIF on 25 Feb 12 15:23 in step 3, but not how.

I can't read your mind, nobody can.

Bye, Olaf.

 

[stanlyn]

Sorry, but I thought the line that starts the number list you referred to makes it clear that the process included the conversion routine.

For every record, it... (it being the routine)

Here is the HOW which makes little sense to most...

lnSlaHandle = Fopen(Alltrim(DOCUMENTS.image_path))

If lnSlaHandle > 0
* Seek to end of file to determine the number of bytes in the file
nSlaFileSize = Fseek(lnSlaHandle,0,2) && Move pointer to EOF
If nSlaFileSize <= 0
If Not Deleted()
goThisform.PageFrame1.pagTasks.edtStatus.Value = ;
goThisform.PageFrame1.pagTasks.edtStatus.Value + ;
'Bad Sla Image for Page: ' + lcSlaFileName + Chr(13)

goThisform.PageFrame1.pagTasks.edtStatus.SelStart = ;
Len(Alltrim(goThisform.PageFrame1.pagTasks.edtStatus.Value))
Inkey(.01)

llBadSlaImage = .T.
llBadTifImage = .T.
Endif
Else
= Fseek(lnSlaHandle,0,0) && Move pointer to BOF
lmSlaBinary = Fread(lnSlaHandle, nSlaFileSize)
= Fclose(lnSlaHandle) && Close the file

lmTifBinary = ''
llBadTifImage = .F.

With goThisform.oleLead1
pcOldErrorHandler = On('error')
lcMessage = ''
On Error lcMessage = Message()

.EnableMethodErrors = .F.
.Load(DOCUMENTS.image_path,0,0,1)

* Error 1440 - OLE Exception
If 'OLE object' $ lcMessage
goThisform.PageFrame1.pagTasks.edtStatus.Value = ;
goThisform.PageFrame1.pagTasks.edtStatus.Value + ;
'Bad Sla Image for Page: ' + lcSlaFileName + Chr(13)

goThisform.PageFrame1.pagTasks.edtStatus.SelStart = ;
Len(Alltrim(goThisform.PageFrame1.pagTasks.edtStatus.Value))
Inkey(.01)

llBadSlaImage = .T.
llBadTifImage = .T.
On Error &pcOldErrorHandler
Else
If .Bitmap <> 0
m.pg = 0 && single page files
.GetFileInfo(DOCUMENTS.image_path, m.pg)
m.fmt = .InfoFormat
m.Width = .InfoWidth
m.Height = .InfoHeight
m.bps = .InfoBits
m.xres = .BitmapXRes
m.Yres = .BitmapYRes
m.pg = .InfoPage
m.sizd = .InfoSizeDisk
m.sizm = .InfoSizeMem
m.cmp = .InfoCompress

lnStatusLen = Len(Alltrim(goThisform.PageFrame1.pagTasks.edtStatus.Value))
lcMessage = ''
pcOldErrorHandler = On('ERROR')
On Error lcMessage = Message()

goThisform.PageFrame1.pagTasks.edtStatus.Value = ;
goThisform.PageFrame1.pagTasks.edtStatus.Value + ;
'TifImage: Exported' + Chr(13)

* Error 1903 - String is too long to fit.
If 'too long to fit' $ lcMessage
lcMessage = ''
Do 'WriteStatusFile' With (pnParts + 1)
Retry
Endif

On Error &pcOldErrorHandler

goThisform.PageFrame1.pagTasks.edtStatus.SelStart = lnStatusLen
Inkey(.01)
Endif

Do Case
* 6 = FILE_BMP
* 29 = FILE_CCITT_GROUP4
* 26 = FILE_LEAD1
Case m.bps = 1
.Save(m.pcTifFileName, 29, 1, 1, .F.)
Case m.bps = 2
.Save(m.pcTifFileName, 29, 2, 1, .F.)
Case m.bps = 4
.Save(m.pcTifFileName, 29, 4, 1, .F.)
Case m.bps = 8
.Save(m.pcTifFileName, 29, 8, 1, .F.)
Case m.bps = 12
.Save(m.pcTifFileName, 29, 12, 1, .F.)
Case m.bps = 16
.Save(m.pcTifFileName, 29, 16, 1, .F.)
Case m.bps = 24
.Save(m.pcTifFileName, 29, 24, 1, .F.)
Case m.bps = 32
.Save(m.pcTifFileName, 29, 32, 1, .F.)
Otherwise
.Save(m.pcTifFileName, 29, 1, 1, .F.)
Endcase

.Bitmap = 0
= Fclose(lnSlaHandle) && Close the file

pcTifFileName = Left(Alltrim(DOCUMENTS.image_path), (Len(Alltrim(DOCUMENTS.image_path))-4)) + ".TIF"
lcTifFileName = pcTifFileName
lnTifHandle = Fopen(lcTifFileName)

If lnTifHandle > 0
* Seek to end of file to determine the number of bytes in the file
nTifFileSize = Fseek(lnTifHandle,0,2) && Move pointer to EOF
If nTifFileSize <= 0
goThisform.PageFrame1.pagTasks.edtStatus.Value = ;
goThisform.PageFrame1.pagTasks.edtStatus.Value + ;
'Bad Tif Image for Page: ' + lcTifFileName + Chr(13)

goThisform.PageFrame1.pagTasks.edtStatus.SelStart = ;
Len(Alltrim(goThisform.PageFrame1.pagTasks.edtStatus.Value))
Inkey(.01)

llBadTifImage = .T.
lmTifBinary = ''
Else
= Fseek(lnTifHandle,0,0) && Move pointer to BOF
lmTifBinary = Fread(lnTifHandle, nTifFileSize)
= Fclose(lnTifHandle) && Close the file
Endif
Else
llBadTifImage = .T.
lmTifBinary = ''
Endif

.ForceRepaint()
Endif

On Error &pcOldErrorHandler
Endwith
Endif
Else
goThisform.PageFrame1.pagTasks.edtStatus.Value = ;
goThisform.PageFrame1.pagTasks.edtStatus.Value + ;
'Bad Sla Image for Page: ' + lcSlaFileName + Chr(13)

goThisform.PageFrame1.pagTasks.edtStatus.SelStart = ;
Len(Alltrim(goThisform.PageFrame1.pagTasks.edtStatus.Value))
Inkey(.01)

llBadSlaImage = .T.
Endif

Thanks, Stanley

 

[Olaf Doschke]

the conversion routine."

You just talked about conversion, not which routine, so A, not THE conversion routine. And you can convert to tif in many many many ways, including some NOT visually displaying, but just working on the bytes, you can use GDIPLUS or any EXE converting from->to any format not needing to really process it in a way to get memory access failures, eg conversion routines may detect that wrong image data would lead to such access violations and deny converting.

What ole control is goThisform.oleLead1? without this knowledge your conversion will stay a mystery. I see the ole controla has a Load method, by .Load(DOCUMENTS.image_path,0,0,1), loading an image.

Going back to your initial post: syncdri.exe is your VFP compiled exe or something coming from third party? Do you know what commands are used within and what modules or ocxes or dlls it uses? You can find out using dependency walker, just google "dependency walker".

Bye, Olaf.

 

[Olaf Doschke]

And while we're at tools, you could use process monitor to monitor syncdri.exe and see at the end of the log what it did last before crashing.

One more general observation I made is that since Vista some C5 errors that in XP also are labeled "fatal excpetion" and show up as memory violation. If that is the case you may really just be facing either a corrupted image OR dbf, eg your Documents.dbf.

Bye, Olaf.

 

[stanlyn]

>> What ole control is goThisform.oleLead1
goThisform is a global object form, while oleLead1 is a Leadtools main image activex control from

http://www.leadtools.com

>> Do you know what commands are used within and what modules or ocxes or dlls it uses?

Yes, the syncdri.exe is my exe that contains the leadtools activex imaging controls.

What I need is a way to see exactly what was being done at the time of failure, and if the process monitor records grainular details and doesn't buffer, then that may shed some light.

Just wondering if the coverage log would work? Is that data buffered which would be lost in a failure, or I can insert a bunch of wait key messages with desc of where its at...

I'll get back to testing these methods when sql server gets its stuff done...

Thanks, Stanley

 

[Olaf Doschke]

Well, as far as I know, but haven't put to a hard test is COVERAGE outputs while excuting. Whatever is in buffers, is in some OS or file system buffers not crashing, if syncdri.exe crashes.

Processmonitor also is "live logging" as far as I know. I knew one or two tricks with it, you may be able to let it begin loggin, if a file it looks out for is created, so the vfp exe can siply fcreate() a file to trigger process monitor to watch out from then, otherwise the log can get large, as with coverage.

The thing with process monitor is, it is now one tool previously there was a seperate file monitor, you can see what API calls were made last, even perhaps from the leadtools ocx and what files where processed, which would help in both cases some image file or the dbf file is the culprit.

Bye, Olaf.