Working With BIG Char Strings

[JRB-Bldr]

I am attempting to use

  mcString = FILETOSTR(MyZipFile.zip)

to convert a file into a string so that I can convert a few characters within the file.

The following general approach worked well on a small file, but on a BIG file I ran into problems.

For this specific test the FILETOSTR resulted in a 22820066 byte string (a typical file size for production runs). And this worked well so far and it accepted the BIG file into the variable with no problem.

But when I attempt to change the variable contents with something like:

  mcString = STUFF(mcString,22820019,1,CHR(01))

I got an error message:
String too long to fit

So I took another approach...
I next tried to segment the string into parts

  mcLeftString = LEFT(mcString,22820018)
  mcRightString = SUBSTR(mcString,22820020)
  mcNewChar = CHR(01)
  mcString = mcLeftString + mcNewChar + mcRightString

But I again got the error message:
String too long to fit

Yes I could most likely do the whole thing with Low Level File operations, but this was a much quicker approach.

Any suggestions on how to approach changing a character or two within the BIG string so that I can use the STRTOFILE() to re-write it back in its modified form?

Thanks,
JRB-Bldr

 

[bborissov]

With BIG file low level functions is veery fast:
(example is w/o any error handling)

lnHandler = FOPEN(.....)
FSEEK(lnHandler,22820019,0) && Maybe it must be 22820018
                            && I am not sure FSEEK is zero based
                            && or 1 based
FWRITE(lnHandler,CHR(1),1)
FCLOSE(lnHandler)

Borislav Borissov
VFP9 SP1, SQL Server 2000/2005.
MVP VFP

 

[NasibKalsi]

VFP 9.0 Char vaiable Limit is 16,777,184 (2^24 - 32). So do not use any pre defined string functions which involve string handling above the limit(subst, left, right, etc.)

I think there is a bug in the string handling, and allows to read in a Char Var above the limit. Also you can do string operation(concatenate) above the limit. I do not know exactly where the bug is, but certainly there is one.

To solve the problem:

you could use

c1 = subst(mcString,14000000)
c2 = subst(mcString,14000001)

Now use only c2 to manipute your string data as required. Make sure the len(c2) < (2^24).

and join all substrings.
and strtofile()

try it.

Nasib Kalsi

 

[NasibKalsi]

sorry

c1 = subst(mcString,14000000) && or use left()

should read

c1 = subst(mcString,1,14000000)

Nasib Kalsi

 

[JRB-Bldr]

Thanks for the replies.

With some modifications to the code, the suggestion from Borislav did what I needed reasonably well.

Thanks,
JRB-Bldr

 

[NasibKalsi]

Jrbbldr:

I thought, you do not like to use low level function(s) for this operation.


Nasib Kalsi

 

[JRB-Bldr]

What I was trying to avoid was doing a full file parse using low level functions. While I have created more than a few low level text parsing routines for EDI & XML processing, etc., they are time-consuming to create and, for BIG files, they can work relatively slowly.

Borislav's suggestion pointed me in the right direction to quickly target into the specific byte(s) that I needed to change. Quick in and Quick out -- Done!

Thanks,
JRB-Bldr

 

[dbMark]

VFP 9.0 sp1 specifications:

Help file for FILETOSTR()- "The amount of available memory or disk space determines if you can store the character string to a memory variable, array element, or memo field."

Help file for Visual FoxPro System Capacities - "Maximum # of characters per character string or memory variable... 16,777,184"

I can use FILETOSTR() and STRTOFILE() to read into and write from a memory variable, just now tried it with an 18 MB file which exceeds the above limit.

I know in the past I tried using FREAD() to read in 65K chunks of large files and it would crash at about 10 or 12 MBs. Maybe I was encountering that 256^3 character count limit? That's when I switched to FILETOSTR() and STRTOFILE() and had no further problems. I used it for copying files that were hidden for security reasons from folder directory listings.

I tried changing some data at the end of the string with STRTRAN() and CHRTRAN() and it worked fine. That said, there may be some limits on certain groups of string-handling functions but I'm unclear at this point.

VFP sp1 (soon sp2) rocks!