Project Compiler suddenly stopped including newer files 3

[Joe Crescenzi]

Hi Everybody,

I've been developing xBase programs since dBase II in the early 80s and all of a sudden I'm having the craziest problem, the project manager is skipping files on my most critical project and I have no idea why.

The strange this is that this is the same program I've been continuously developing since 1985, and I've compiled newer versions of it at least once a week without a single problem.

Now, when I compile the program, all the newer PRG procedures, and SCX forms are NOT in the EXE file... and are NOT in the Project File.

I'm wondering if anyone else has seen this problem before.

I assumed the project file was corrupt, so I created a new one and just added the main PRG file, expecting it to rebuild the whole project, but it didn't. In fact based on opening the project file directly as a table, the newer project file literally skipped nearly 600 files that otherwise were in the original, and it's basically saying in the error file that the files were not found. All the files are in the same directory as the project file.

I even tried compiling it on an entirely different computer, so it's not a corrupted version of VFP.

I'm starting to wonder if the compiler somehow hit some sort of limit to the number of files in either the directory or the project file itself?

There are over 4400 files in the directory, including about 1400 PRG files, 400 SCX files and a ton of report and other files.

My workaround so far was to add some of the missing files by hand to the project file, but I worry that I won't know which files are missing without either waiting for a customer to say "there's an error saying procedure xys is not found".

I'm thinking of writing a script to look at all the PRG and SCX filenames that are in the director, but not in the project file so I can know which ones to add, but that's not not the best option because there are other prgs and scx files that aren't actually called, so I'd need to start maintaining my own list of files that can be excluded.

This problem really has me confused. I've been developing code for 40 years ands it's like somebody suddenly telling me up is now down, or blue is now red. There's just no reason for the project to think files are missing when they're right in the same directory.

Any thoughts?

 

[Steve Meyerson]

Hi Joe,

You say the 3 prg's are short. I know this makes no logical sense, but maybe recreate/retype them to replace the originals.

I suggest this because on rare occasions, (maybe 2 in several years) I have had to retype a perfectly good statement to avoid a compilation error.

Just a wild guess.

Steve

 

[mJindrova]

Joe, how much branches has DO CASE... ENDCASE in main prg?


mJindrova

 

[Joe Crescenzi]

********* I FOUND THE PROBLEM
*********
********* DRUMROLL
*********
**
** It was the way XFRX was (accidentally) used.
**
*********

Many years ago, before I used XFRX to create PDFs using a ReportListener, I called a function that was part of an older XFRX library that didn't create PDFs from reports, but let you basically place text on the page programatically. It was close to what I did in the DOS days, which was to write PostScript code, which turns out to be pretty easy once you get the basics. You give it a string followed by coordinates starting from the lower left as 0,0 and told it to show it there. I loved the control I had, but when the ReportListener was developed and XFRX used it, I stopped using the old way.

To use it, you referenced it with "SET PROCEDURE TO XFRX" so it could find the classes whenever you needed to call those functions. Apparently, one of those remaining prgs still had a SET PROCEDURE to XFRX in the code.

To fix it, I just commented out that one line and I can now build a perfect project with everything.

I don't even remember how long ago I used XFRX as a procedure file, but there it was, and for some reason it freaks out the compiler if it's anywhere in the code.

I confirmed this multiple times in my sandbox. When I put that one line in the code it gives me the tiny PJX, when I remove it I get all 1400.

I thought for sure I eliminated XFRX from my testing because I did at least one test with the files removed, but I guess I missed it after all.

It was fun exploring all the possible options... thank you for taking the time to help me figure out this puzzle.

In the end, there's still one tiny mystery... Why would the compiler freak out when there's a reference to a PRG that otherwise works when you call it the regular way? If anything it should've added it, and the runtime would've barked over that unnecessary reference.

 

[Chris Miller]

Why would the compiler freak out

Indeed, as I already said any code that's compiled isn't executed at the same time, so how could it trip up the build process at all?

You could do one more experiment. Uncomment the SET PROCEDURE, but before you build issue a CLEAR ALL.

Chriss

 

[Joe Crescenzi]

Indeed, as I already said any code that's compiled isn't executed at the same time, so how could it trip up the build process at all?

I have one theory, but it's a stretch.

As I mentioned, I licensed it with source, which means the folder has XFRX.PRG and XFRX.APP.

It could be that the SET PROCEDURE TO XFRX triggered the compiler to crawl the PRG, even though there's already an APP in the folder and it became sort of a Clark Kent / Superman thing where the two should not exist in the same place. When it saw the same internal objects in both places it bailed out. It could even have something to do with the recent update I did, so the APP might be newer (or older) than the source.

In either case, I'll keep them apart and only compile a new APP if I decide to alter the code (Which isn't likely, or I'll need to re-mod every time Martin updates it).

Above all I made sure there are no longer references to the pre ReportListener way of declaring XFRX a procedure file.

 

[mJindrova]

Joe,

Please, delete xfrx.app - it's a demo version.

https://eqeuscom.atlassian.net/wiki/spaces/DOC/pages/3899411/Frequently+asked+questions+-+FAQ

2.1. I am evaluating XFRX. It works fine in development, but when I try to compile the demo in my project, the VFP quits or crashes. Is that because it is a demo version?
This is an issue in the demo version of XFRX - it cannot be included into VFP projects, it makes VFP crash (this is a negative side effect of a decompilation protection). The workaround is to invoke XFRX via macro substitution:
loSession = EVALUATE[XFRX('XFRX#INIT')]
This way XFRX.APP does not get into the project and you will be able to compile without problems. The commercial version is not affected by this problem.



mJindrova

 

[Chris Miller]

Thanks, mJindrova.

That explains a lot.

Aside of that, Joe, why would you let an app file be part of a project? I don't see any case where an APP would be included in a build and work embedded into a final EXE. Take the usual report engine, not even foxypreviewer, you're asked to deploy the app files separate to your EXE as part of the runtimes. So even if an app would not cause build problems, it should always be kept separate. The only reason to include an app in the project itself, but excluded, is that it doesn't cause unresolved references.

That adds to the build from scratch only having the main.prg in the project before building: You either also add in the app file and set it to excluded before the first build or you keep out any references to the APP file from any code, so it's not included in the build. No matter if there are full sources included or not.

Chriss

 

[Gerrit Broekhuis]

Joe,

Glad you found the problem.

I never include an app file in a built, it’s allways distributed separately.

Regards, Gerrit

 

[Joe Crescenzi]

I've never included the App implicitly in my Project because I've always included the App externally in my distribution.

The project worked perfectly until I somehow brought in an older PRG that had a "SET PROCEDURE TO XFRX" line in it.

The project file has worked for decades, but one of the 3 last DOs in my main program had that line because the version of XFRX created before VFP introduced the ReportListener class was done via a "Set procedure to XFRX".

For what it's worth, once I removed that one line, I can build my project without it skipping anything.

In summary, with the Set Procedure To XFRX line there, it breaks the compiler. The earlier that line is in the program, the more files it would skip. Removing it works perfectly.

Meanwhile, I recently downloaded an even newer version of XFRX, so I'll be very careful to remove the old one and I know for sure I don't have any other references to XFRX using the older SET PROCEDURE method.

 

[Chris Miller]

Joe,

a SET PROCEDURE TO something does not only address something.prg, it also adddresses something.app and includes an APP file if one is found. You can check it out with a new project that has a main.prg which does nothing else but

set procedure to reportoutput

.
You'll see that building that will include reportoutput.app from HOME() into the project, excluded is automatically set.

The rest then is obvious with mJindrova's explanation.
And since excluded is the default I think the build problem also occurs in that case. Indeed when you open the debugger before build and open the debugout window, you can see VFP processes excluded files, too. It's not a propblem with any app, so you have to know it's caused by the copyright protection of XFRX.

Just for sake of demonstrating what happens if an app is included in the build process: Create a new empty project, create a new main.prg with just SET PROCEDURE TO REPORTOUTPUT.
open up the debugger with Tools->Debugger and open the debugoutput window.
Build the project and debugoutput should output this:

Build Project: Program main
Analyzing: MAIN
Analyzing: FRXOUTPUT
   Module: ReportOutputCleanup
   Module: TestListenerReference
   Module: GetSupportedListenerInfo
   Module: ReportOutputConfig
   Module: GetConfigObject
   Module: ReportOutputDeclareReference
   Module: UnloadListener
   Module: HandleError
   Module: CheckPublicListenerCollection
   Module: Execute
Build Application: Program main
Build Application: Application reportoutput

You see:
1. even though reportoutput.app is included into the project as excluded project item, it is analyzed.
2. All the lines after Analyzing: MAIN are about reportoutput.app and you can see VFP goes into modules of the reportoutput.app.
All that's also happening with XFRX.app once that's included into the project. And I guess one of the last lines in the debugout of your failing build would have been about a module of XFRX.app.

Notice how hard that is to detect even in this well known case: The line after analyzing main is not stating to analyze reportoutput.app or reportoutput, it says:

Analyzing: FRXOUTPUT

So it may not become clear the build hangs on XFRX.app

It becomes clearer once you look into the reportoutput.pjx coming with XSOURCE.ZIP. In that project the main program is called FRXOUTPUT.PRG, so that's what's analyzed from an APP file, the embedded main file of the app. It's not easy to spot and you need to know the sources of an app to realize that. That needs to be known on top of knowing that SET PROCEDURE TO xyz also addresses xyz.app, which is not mentioned in the help topic of SET PROCEDURE. So all in all that's it, but even if I knew what mJindrova knows about the source code protection of XFRX. If the XFRX.app main file also is main.prg it becomes even worse, the debugoutput would state to analyze main once more and you never would suspect that means main.prg inside XFRX.app, but who knows, I don't have XFRX to check it out. The cherry on top is that you stated XFRX can't be the problem since you excluded usage of it. Well, obviously you didn't remove it completely.

Chriss

 

[Joe Crescenzi]

The cherry on top is that you stated XFRX can't be the problem since you excluded usage of it. Well, obviously you didn't remove it completely.

I did so many experiments and I thought I tried that at least once, but several people mentioned that XFRX wasn't likely the cause because it's been well tested over the years. I just didn't realize one of my PRGs had that set procedure to xfrx in it, or that it would do anything other than reference the same files it already referenced.

For what it's worth, I'm going to do a deep dive as to the latest XFRX docs to see what's changed over the years. I've used it for many years without problems, but there could be new best practices that I'm not aware of in the docs because I haven't needed to read them in a very long time. Who knows, maybe the latest version has new things I should explore.

I'm also going to put a preferred report viewer setting into my system so I can swap between multiple report listeners to see which ones work best, or even let my customers choose which ones they like on the fly.

Right now all my reports are passed through a single function I simply call RepView.prg, so I should be able to do a simple do case without having to modify hundreds of potential places where reports are called. Then I can easily try other report viewers and pdf generators like the newer FoxyPreviewer, which I couldn't do previously because I have a high DPI screen (the older version would mess up the page size if you have a high DPI monitor).

 

[Joe Crescenzi]

mJindrova (Programmer)18 Mar 24 19:58
Joe,
Please, delete xfrx.app - it's a demo version.

2.1. I am evaluating XFRX. It works fine in development, but when I try to compile the demo in my project, the VFP quits or crashes. Is that because it is a demo version?
This is an issue in the demo version of XFRX - it cannot be included into VFP projects, it makes VFP crash (this is a negative side effect of a decompilation protection). The workaround is to invoke XFRX via macro substitution:
loSession = EVALUATE[XFRX('XFRX#INIT')]
This way XFRX.APP does not get into the project and you will be able to compile without problems. The commercial version is not affected by this problem.

You are correct, the demo version as distributed as an APP. The live version is distributed as FXP by default, but if you pay extra for the source, it's a PRG. I use the source version, so I have a PRG, but I compiled mine to an APP, so my APP is a live version, although technically, if I delete the APP the compiler will add the PRG into my project and it should work the same, but make the exe a bit bigger. It's already 14MB, so it wouldn't be a big change.

Here's how I call it in my report viewer PRG.

  loxfrx = XFRX("XFRX#LISTENER")
  loxfrx.targetType = "PDF"
  loxfrx.targetFileName = MyFileName
  lnRetval = loxfrx.SetParams()
  IF RECCOUNT() = 0
    VovioMessage("This report is empty.  Try changing the filters")
    RETURN ""
  ENDIF
  IF lnRetval = 0
    REPORT FORM (_RepFile) OBJECT loxfrx
    loxfrx.finalize()
    RETURN MyFileName
  ELSE
      ? "Invalid output folder/file", MyFileName, lnRetval
      WAIT WINDOW
  ENDIF

 

[mJindrova]

Joe,


if you compile xfrx to separtly app, then the same rules apply as for the demo.

mJindrova

 

[Joe Crescenzi]

if you compile xfrx to separtly app, then the same rules apply as for the demo.
mJindrova

Then that makes it even easier for me to decide what to do. I can just delete the APP like you said and just call the PRG.

For what it's worth, if that's the case, it's always been pulling the PRG into my project and the runtime most likely ignores the APP.

The other thing I've learned is that the SET PROCEDURE TO XFRX is to be avoided at all costs.

 

[Chris Miller]

I don't use XFRX, but now I'm interested, too.

if you compile xfrx to separtly app, then the same rules apply as for the demo.

But you also said:

...XFRX.APP does not get into the project and you will be able to compile without problems
...
The commercial version is not affected by this problem.

So I'm still not getting this straight: Does or doesn't the commercial XFRX.app make VFP crash when building a project?

Also, from my side, Joe:
Since SET PROCEDURE still prioritizes PRGs over APPs, it does not necessarily cause the XFRX.APP to become included into your project. That must have been something caused by files available and not available and seen by the build process. So, in the end, if only the demo app causes build crashes, you could still have SET PROCEDURE TO XFRX to either cause including XFRX.prg or XFRX.app or both, not sure how that would turn out. Because even if a SET PROCEDURE TO XFRX.prg would explicitly only include the prg file, if that in turn contains code that uses the APP without the macro substitution workaround, it would include the APP anyway.

Now, it only depends on your clarification, mJindrova. Is this really only a demo version problem or to be avoided generally?
And on top of that, would even direct usage of a prg, not the app, include files eventually causing the build problem, so you don't just have to avoid SET PROCEDURE TO but any direct usage? I can't imagine tha latter to be "user" (=programmer) friendly.

Chriss

 

[Joe Crescenzi]

Now, it only depends on your clarification, mJindrova. Is this really only a demo version problem or to be avoided generally?
Chriss

I was wondering the same thing. I was planning on making another APP like I did in my older version, but now I wonder whether that file was being ignored (because I also had xfrx.prg in the same folder)... or used. If using it as an APP makes it a demo, then I have no choice.

Although I've renewed my source code version multiple times over the years (mainly to support an independent developer), I've apparently been using a 2010 build because I never quite got the source to work on newer versions and I completely forgot which one was actually being called.

So, in the build that has been working for me (using an APP I compiled in 2010), there are no errors as long as I don't use the dreaded set procedure to xfrx.

Meanwhile, as I explore my working folder, I decided to finally upgrade to the latest version (Feb 2024)... and wouldn't you know it... the project file is a train wreck again.

The program runs flawlessly after adding about 30 files, a mixture of files like DLLs, FLLs, VCX, etc.

Compiling it to a fresh Project yields a bunch of things, starting with this:

Program c:\sourcecode\cartpro.new\xfrx.prg has the following errors:
    Unknown _XFGETLTI - Undefined
    Unknown _XFBMP2 - Undefined
    Unknown _XFCROPIMAGE - Undefined
    Proc./Func. XFRX_EXTRACTFILEFROMAPP - Undefined
    Proc./Func. XFRXPROXY - Undefined
    Program XFRXPROXY - Undefined
    Proc./Func. XFRX_TESTFILEINAPP - Undefined
    Proc./Func. XFRX_USETABLEFROMAPP - Undefined
    Proc./Func. XFRX_SETPICTINAPP - Undefined
    Unknown _XFCONVERTIMAGE - Undefined
    Unknown _XFIMAGEBPP - Undefined
    Unknown _XFWMF2IMAGE - Undefined
    Unknown _XFWID - Undefined
    Unknown _XFDEC - Undefined
    Proc./Func. XFRX_EXTRACTREPORTFROMAPP - Undefined
    Proc./Func. XFRX_RUNMACROINAPP - Undefined
    Proc./Func. XFRX_RUNPROGRAMINAPP - Undefined
    Unknown _XFIMAGES - Undefined
    Unknown _XFGETVERSION - Undefined
    Unknown _XFIMAGEGETDPI - Undefined

Meanwhile, after that bunch of errors, it starts say about 150 or so of my PRGs are undefined again, so the project loses another 150 prgs again. It's not as bad as only having 252 records, but clearly not an easy update from the 2010 version that generated no undefined objects just prior to trying to update it.

So, if the solution is to generate an APP first, like I did in 2010, I really need to know whether that means that the code runs like a trial version when made into an APP or not, because I just want it to work as it did before.

 

[Chris Miller]

I think compiling an app from full source should not create an app that crashes VFP. In general, you can have apps in projects without getting build problems.

Then what still happens is when you call prgs from an app not included into your own project, you get undefined error messages, but can ignore them, as they are contained in the external app. To not get compile errors you'd define things in a #IF .F. #ENDIF section to keep the parser silent about undefined functions that are still present in the app.

Or you go the completely opposite direction and don't build an APP, instead include the XFRX sources into your own project so they become integral part of the EXE.

mJindrova,
I still look forward for clarfications and advice from you.

Chriss

 

[Joe Crescenzi]

It doesn't bother me that the compiler considers all those objects undefined.

What bothers me is that now the compiler is missing over 150 of my own files again, and will likely stop bringing in new stuff just like it did when this whole thing started.

So, unless I figure out how the newest XFRX needs to be integrated, I'll go back to the 10+ year old version and put my code back the way it was... except for that one line that broke the compiler in the original.

 

[Chris Miller]

missing over 150 of my own files again

To me the list looks like only mentioning XFRX components. You know best what's missing from your own files, of course.

I wonder why you would even need to include anything from XFRX to call something in an APP. Or in the alternative case, when you do loxfrx = XFRX("XFRX#LISTENER") why you would build an APP at all. There should be documentation about how to use XFRX, shouldn't there, also from the point of view of what files to add to your own projects, if any.

Chriss

 

[mJindrova]

https://eqeuscom.atlassian.net/wiki/spaces/DOC/pages/3899432/Running+XFRX

Important note
The evaluation version of XFRX cannot be included into VFP projects, it makes VFP crash. To avoid this please invoke XFRX via macro substitution:
[pre]
LOCAL m.loSession
m.loSession = EVALUATE("xfrx('XFRX#INIT')")
[/pre]
This way XFRX.APP does not get into the project and you will be able to compile your application without problems.

RESULT: If Joe compile xfrx.prg to xfrx.app then must initiate XFRX object follows: loxfrx = EVALUATE("XFRX('XFRX#LISTENER')")


mJindrova