ReusableStreamDecode plus SubFileDecode syntax

[DivByZeroSetBBox]

I've been through some of the threads on using large data sets that can't be described in a single string. I wanted to be able to use the PostScript level 3 syntax with ReusableStreamDecode and SubFileDecode in order to print some large RGB images, and have been unable to figure this out. From what I can tell, I have the right syntax, but it fails either with ps2pdf, gs, or a level 3 Xerox printer. I minimized it down to the following code, which seems like it should work, but does not:

%!PS

/ImageData
currentfile
<< /Filter [/SubFileDecode /ASCIIHexDecode]
/DecodeParms [<< /EODCount 0 /EODString (*EOD*) >> null]
>> /ReusableStreamDecode filter
ffddccffddccffddccffddccffddccffddccffddccffddccffddcc
*EOD*
def

/Helvetica [24 0 0 24 0 0] selectfont
72 72 moveto
(TEST) show
showpage


Why does the code above (side-question: How do I format the block above as code on this forum?) fail with:
Error: /undefined in --filter--
Operand stack:
ImageData --nostringval-- --dict:2/2(L)-- --dict:2/2(L)-- --nostringval-- --nostringval-- true ASCIIHexDecode 1

From what I can tell this error should not occur. What do I need to change to make it work?

 

[DivByZeroSetBBox]

I forgot to mention, I'm fairly close to certain that the problem is the syntax of putting the actual data in; that the problem is related to these lines of code/data:
>> /ReusableStreamDecode filter
ffddccffddccffddccffddccffddccffddccffddccffddccffddcc
*EOD*

I have yet to find out with certainty how the beginning of data syntax should look...all I've found for examples show the *EOD* string and not some form of start that works with SubFileDecode and ASCIIHexDecode.

 

[SJProuty]

The code does look like it should work. However, the error seems to indicate a problem with ASCCIIHexDecode. Try terminating the hex data with a ">" without the quotes. This should then satisfy that filter's EOD condition. Also note that because ASCIIHexDecode filter already has a defined terminator, you can probably eliminate the SubFileDecode filter in this case.

Thanks,
Scott Prouty

P.S. You should check out this forum:

http://www.tgreer.com/printforum/

It has a very good PostScript forum, plus it has a very nice advanced editor that makes it easy to do things like code blocks, etc.

 

[DivByZeroSetBBox]

Termination had no effect. In any case, since I'm using this for image data that exceeds normal string length limits, I'm fairly certain that any requirement to use string style termination would also result in exceeding resource limits (the string indexing is limited to 64k, my images are all multimegabyte up to hundreds of megabytes). I'm thinking that because the byte offset in the error message always points at the end of the "filter" token, and because adding more whitespace between filter token and data does not change the byte offset where the /undefined occurs relative to byte offset...that this is related to either the start of data syntax, or else to something prior to data. It seems that it has never reached beyond the first character of data.

Is it possible to read uncompressed RGB hex data for use in large images without hex decoding? My limited impression of the ASCIIHexDecode is that there is a conversion to a binary format; and that I need the SubFileDecode so that this binary format can be fed to the image when data length exceeds string limitations.

One of my attempts at solving this has been to save PostScript via print-to-file of various PostScript-aware applications; none of these resulted in the level 3 dictionary approach, nor in the mix of SubFileDecode and ReusableStreamDecode. Apparently all of these applications stick with level 2. I'd love just one sample of code that actually reads using SubFileDecode and ASCIIHexDecode through a ReusableStreamDecode, but have not found one yet.

I'll gladly check out the other forum too...I probably won't have time to do so till this upcoming weekend...glad to hear of more forums that cover PostScript.

 

[SJProuty]

I just ran your origianl code posted above through Adobe Distiller 6.0 and it ran fine with no errors. Are you sure the interpreter you are testing with is Level 3?

As for the ASCIIHexDecode terminator ">", note that that is just a termination sequence for the filter, as if you had done something like:
<< /EODCount 0 /EODString (>) >> /SubFileDecode
It has nothing to do with hex strings and therefore does not have string size limitations.

Here is the original code reworked to remove the /SubFileDecode filter, which is not needed for hex data:

%!PS

/ImageData
currentfile
<< /Filter /ASCIIHexDecode >> /ReusableStreamDecode filter
ffddccffddccffddccffddccffddccffddccffddccffddccffddcc>
def

/Helvetica [24 0 0 24 0 0] selectfont
72 72 moveto
(TEST) show
showpage

Note that in this case, the hex data can be megabytes and megabytes of data, the ASCIIHexDecode filter will read it. All it requires is a single ">" at the end of all the hex data.

Thanks
Scott Prouty

 

[DivByZeroSetBBox]

When actually using this, it fails with a rangecheck error.

Keep in mind that the real data exceeds 64k by quite a bit, and a string can only be indexed to 64k (RGB 8 bit per channel implies six times what a string can handle for hex on 64k pixels...that end ">" means the interpreter is counting till it reaches that...at one beyond 64k it blows up).

What it comes down to is that the non-SubFileDecode version *must* read the entire string when the def is used. With SubFileDecode, the data can be returned in usable sizes. From the PostScript Language Reference, 3rd edition, page 153:
"Such data can be stored in strings, but only if the amount of data is less than the implementation limit imposed on string objects....To overcome this limit, LanguageLevel 3 defines reusable streams."

Well, so far as I know, all implementations limit strings to 64k or less.

Someone might doubt ghostscript is level 3, but it claims it is. If that isn't good enough, then the high end Xerox-Tektronics printers I'm using which are true Adobe PostScript 3 (and probably the best print engine there is for PostScript) agree with ghostscript, and have the same error (and we're talking about printers with half GB of ram, and quite a few GB of hard drive virtual swap space).

Now even if I could get the non-SubFileDecode version to work, the definitions given in the reference book for the original mixture are correct. What the reference books fail to do is show how the syntax for starting the data should occur after that mixture (I'd love a book that is a true in-depth book on PostScript...alas, not even the Adobe publications qualify for this, including green, red, or blue books...the red book comes closest, but misses).

I still have more things to try this weekend, but at the moment it's looking pretty hopeless. I see definitions of level 3 features, but so far nothing I've found in the real world actually uses those features such that I can apply it.

 

[SJProuty]

Remember that both the code I posted and the code in the original post ran fine through Adobe Distiller without any erorrs producing a valid PDF. So Adobe Distiller at least does not agree with the other interpreters that are getting an error.

Just for kicks, I tried the original code on two printers. First was an HP2200, which is level 2. Interestingly, it got the same error as in the original post, i.e. Error: /undefined in --filter--. I then sent this to my Xerox DP75, which is level 3. It printed fine with no errors. Hmmm...

The data following the filter is just data. It is not PostScript strings. Note that the following is a PostScript hex string:

<ffddcc>

The following is not:

ffddcc>

An example of using ASCIIHexDecode is shown on page 311 PostScript Language Reference, 3rd Edition. Note that the example says that 393,216 bytes of hexadecimal data is used, thus exceeding the 64K limit for strings.

Thanks
Scott Prouty

 

[DivByZeroSetBBox]

The page 311 version is limited to 64k (65536) samples (it's a composite object). So perhaps the index reaching the limit is within some part other than the string. Still, 64k samples won't work, e.g., some of the data represents 8-bit/channel and 6000x4800 samples. The smaller images are perhaps 3 million samples.

The printers I'm printing on are all high-end true Adobe PostScript level 3...I'm guessing the HP 2200 is an emulation. The Xerox/Tektronics printers are about the best quality interpreter there is, and they are guaranteed to be 100% compliant with level 3. I don't see myself switching to a non-compliant printer...what I really want is to find how to write compliant code. Adobe Distiller is known to provide values as defaults in places where the original code was less than perfect...the code which distiller accepts includes some non-compliant code when it can guess correctly (useful in the real world, not so useful as a syntax checker).

The above code will not work with over 64k samples on a level 3 interpreter. Regardless of this, I want to know why the SubFileDecode will not work, as this is almost certainly some minor syntax thing. Solving the original question would fix quite a few headaches for me...and the original code should work.

 

[SJProuty]

I agree that answering the original question is the way to go. That is what I was attempting to do. However, as I said, of the two level 3 interpreters available to me (Adobe Distiller 6.0 and a Xerox DP75), both run your code in the original post with no errors. The syntax, structure, etc. work fine. So there is not much I can do to debug it further on my interpreters.

I cannot say why your original code gets the error on the interpreters you are using. I tried to sugggest that the interpreters were either not level 3, or do not support the ReusableStremDecode filter. You suggest it is not a problem with the ReusableStreamDecode filter, but a problem with the SubFileDecode syntax. Since you have much more knowledge about the interpreters you are using, I will accept that proposition. The SubFileDecode syntax looks fine and indeed does work on my interpreters.

What I would suggest to you is to separate the filters entirely and test on your interpreters. So, create a file that just has SubFileDecode for more that 64K of data and create another file that just has ReusableStreamDecode filter with more that 64K of data. If you can get those working, then try the combine.

I am sorry I confused the issue by trying to suggest an alternative.

Thanks
Scott Prouty