Export to text file with line length > 256

[radiorog]

Hello. I concatenate a number of fields into one string and export to a text file. The text file apparently won't handle lines longer than 256 characters, and inserts a HR/LF at that point in the text file. I can't tell whether that's a default of VFP or something to do with the maximum length of text lines in Windows.

Any way around this? I really need the output to stay together in one line.

Many thanks for any help.

Roger

 

[Olaf Doschke]

Fwrite supports longer than 256 chars. Also textmerge, \ and \\ do.
What are you doing?

Are you perhaps checking output with notepad with line breaks turned on?

Bye, Olaf.

 

[JRB-Bldr]

I concatenate a number of fields into one string and export to a text file

You have NOT indicated how you are attempting to concatenate your field contents prior to the Write. Are you doing it into a single Memory Variable, or what.

If you concatenate your character fields into an array element and then use STRTOFILE() you can also get more characters out.

Example:

USE MyTable IN 0
DIMENSION aryFlds(1)
SELECT MyTable
nFldCnt = AFIELDS(aryFlds)

* --- Go through the Data Table records in some manner as needed ---
SELECT MyTable

* --- Define Array To Contain Output String ---
DIMENSION aryStr(1)
* --- Populate Output String Array Element ---
FOR FldCntr = 1 to nFldCnt
   cFld = aryFlds(FldCntr,1)
   cFldType = aryFlds(FldCntr,2)
   IF cFldType = 'C'
      aryStr(1) = aryStr(1)  + EVAL(cFld) + ","
   ENDIF
ENDFOR

cDest = "C:\Temp\Output.TXT"
=STRTOFILE(aryStr(1),cDest)

Good Luck,
JRB-Bldr

 

[Mike Lewis]

The text file apparently won't handle lines longer than 256 characters, and inserts a HR/LF at that point in the text file. I can't tell whether that's a default of VFP or something to do with the maximum length of text lines in Windows.

Roger, it doesn't really make sense to say that "the text file won't handle lines longer than 256 characters". A text file is just a file. It contains whatever you put in it.

It's possible that the software you are using to view the text file can't support lines longer than 256 characters, but that's got nothing to do with the file itself.

Also, the insertion of the CR/LF has got nothing to do with VFP, nor with Windows. VFP can handle strings up to 16,777,184. When you write such a string to a text file, it will write the exact contents of the string, without inserting any line-endings or other characters.

In order for us to solve this problem, you will need to tell us what code you are using to write to the text file. If you can show some code that reproduces the problem, I am sure we'll be able to put you right.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips, training, consultancy

 

[radiorog]

Thanks, Mike. Here's what I'm doing. (Pardon the length of this - want to be complete.)

In a VFP8 table I'm using, there are fields of different types - specifically, numeric, date, character, and memo. The routine I use converts all of these into strings, each stored in a separate memvar. When all have been collected for each record, the '?' command outputs the memvars to an 'alternate' text file as a single line. In a sense, the strings are not concatenated in VFP, but rather output collectively as a single text line to the text file.

Example: ? cMemVar1 + cMemVar2 + cMemVar3 + cMemVar4, etc.

The '?' routine also adds some control characters at the beginning, which are used to format the string eventually (see below).

If I open the text file (in Notepad or something like it), I can see that a CR/LF is inserted into the line at a point around 240 'columns' (characters plus spaces). I'm assuming that the control characters must add some length in themselves, which may internally expand the actual byte length of the string that the file sees as totalling 255 (though this is an uneducated guess on my part). Or maybe it's just 'wrapping' the line at a space.

Now, the VFP Memo field can be short or quite long. The CR/LF seems to occur within the memo string portion of the text line, but this may be coincidence. (On a side note, if I set Memowidth to something very large, and display ('?') a long Memo field to the VFP window, it also seems to add a CR/LF at about 200 characters.)

The next step in the process is for a word processor (in this case, WordPerfect) to convert the file to its own format, read the command characters as formatting commands, and create an entry in the list this routine is building. Where the CR/LF occurred in the text file, WordPerfect inserts a Hard Return, thus causing a separate line in the WP file. This occurs whether or not the text file has been previously opened in Notepad. WP also has no apparent limit on line length, so I can only assume it's not the culprit.

So ... the offending CR/LF appears to be added in the process of exporting from VFP to the text file. This is what I'm trying to eliminate.

If it's any help, this table was originally created in FoxPro for DOS. VFP seemed to open it okay, and it operates correctly in VFP. But maybe its origin in FP for DOS has something to do with it?

Any help is greatly appreciated.

Roger

 

[Mike Lewis]

Roger,

This makes perfect sense.

The key point is that you are using the ? command (presumably with SET ALTERNATE TO <text file>). VFP always inserts a hard line-ending in strings when use ?. You would see the same thing if you simply viewed the output on the VFP background screen.

As you have found, the position of the line-ending depends on SET MEMOWIDTH. But, as far as I know, you can't set the memo width to infinity. The best you can do is to set it to its maximum value, which is 8192. If none of your lines will ever exceed that figure, then execute SET MEMOWIDTH 8192, and you should be fine.

But a better option is not to use the ? command at all. Instead, use FWRITE(), or STRTOFILE(). That way, you can be sure that the problem will never arise.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips, training, consultancy

 

[Olaf Doschke]

Yes, using ? is the culprit, the topic on Set Memowidth also says

Note that for ? and ?? the displayed width will not exceed 256 characters.

So even setting memowidth to the max of 8192 does not change this line break at 256 chars.

I already said / and // do not have that limitation, I tried it this way:

Set Textmerge On
Set Textmerge To "d:\test\textmergetest.txt"
lcText = Replicate("a",512)+"z" && double length of what ? can handle plus a "z"
\<<lcText>>
\<<lcText>>
Set Textmerge To
Set Textmerge Off

Adjust the file name to your environment and need. You can check the output file with notepad, but remember it has an automatic line break option and you want to turn that off to see the real file composition.
This would need the least changes to your code. Like ? the \ command adds a line break CRLF at the end, while ?? and \\ do not, instead of var1+var2+var3 you need <<var1>><<var2>><<var3>> with textmerge, but that's about it with the changes, also chr(9) must be turned to <<chr(9)>> to be evaluated and not written as is, because \chr(9) really writes "chr(9)" to the output file. See \ | \\ operator help topic for more remarks on these operators.

I also suggest you use the functions intended for file creation, either STRTOFILE() or FCREATE/FOPEN/FWRITE/FPUTS/FCLOSE. If using STRTOFILE use it's third parameter to either create the file or append to a file.

Bye, Olaf.

 

[Kimed]

I have a situation similar to Roger's and I try to use the following:

SCAN
...
output_string=output_string+field+CHR(13)
...
ENDSCAN

STRTOFILE(output_string, file_name)

Still, lines added from the memo field seem to be truncated at 255. SET MEMOWIDTH didn't have any effect on this.

(I use VFP 6.0, if it matters.)

 

[Olaf Doschke]

STRTOFILE() has no limitation of that kind, it's not even line oriented. It outputs whatever string is the first parameter, so that would surprise me. I don't know if there was a known bug of VFP6, but there were 5 SPs for it.

Bye, Olaf.

 

[Olaf Doschke]

Kimed,

Is "field" A memo? Are there line breaks in the memo itself? You are of course copying over each mome content 1:1 via output_string+field+CHR(13) and add a LF (without adding a CR), after each record.

STRTOFILE() used without the third parameter overwrites the file, but if it already exists, that depends on SET SAFETY. You should be prompted, if you want to overwrite.

Bye, Olaf.

 

[JRB-Bldr]

Kimed - you appear to be using a Memory Variable ( output_string ) to build and contain the string you want to Export.

I'd recommend using an Array element (like I showed above) instead of a Memory Variable.
By doing so you will not experience a character count limitation.

Good Luck,
JRB-Bldr

 

[Mike Lewis]

I'd recommend using an Array element (like I showed above) instead of a Memory Variable.
By doing so you will not experience a character count limitation.

Are you saying that array elements have a higher maximum length than ordinary variables? I thought the upper limit was 16,777,184 in both cases.

I think it's more likely that Kimed's problem is caused by his memo field having embedded hard line-endings, as Olaf suggested.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips, training, consultancy

 

[JRB-Bldr]

Mike - I do see in the Help - System Capacities where it says:
Maximum # of characters per character string or memory variable: 16,777,184

But I have seen too many times when I have attempted to programatically populate a Memory Variable with one or more complex and lengthy SQL Query Command string(s) and things went "belly up".

And when I did the same thing into an Array Element and then use it place of the Memory Variable, I had no problem.
So due to having been 'bitten' too many times - I just go with what has worked for me - for good reasons or not.

Thanks,
JRB-Bldr

 

[JRB-Bldr]

I should add...

for good reasons or not.
Dinosaurs are allowed to do that.

 

[Mike Lewis]

But I have seen too many times when I have attempted to programatically populate a Memory Variable with one or more complex and lengthy SQL Query Command string(s) and things went "belly up".... when I did the same thing into an Array Element and then use it place of the Memory Variable, I had no problem.

Worth keeping in mind. Thanks.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips, training, consultancy