Find all files used in compilation 5

[Vince Wen]

This could be very simple but I am new to VFP9.

I inherited this legacy software that compiles and works fine. But it has a lot of folders and files built up over the years that are not used. I want to get rid of all the unused files so I have a clean slate to start with.

Is there a way to find out what are all the files used in the compilation process? Such that I can delete everything not used?

 

[Mike Lewis]

Welcome to the forum.

I feel there should be some easy way of achieving this goal, but off-hand I can't see what it would be.

However, if you don't come across a better solution, you could try this:

1. Create a new project ([tt]CREATE PROJECT[/tt] or File > New > Project)

2. Switch to the directory containing your application's main program.

3. Add that program to the new project.

4. In the Project Manager window, click Build, then select Rebuild Project and Recompile All Files. Click OK.

5. The Project Manager should not be populated with all the files that are used in the application.

6. From the Project menu, choose Project Info > Files. You will now see a list of all the files (PRGS, forms, reports, etc.) in the project.

This method is far from perfect - mainly because it only shows the file list in a modal window; you probably want to output it to a text file or to print it, which you cannot do in this case. Also, it won't show any files that are referenced indirectly, for example: [tt]lcForm = MainForm / DO FORM &lcForm[/tt]. But at least it's a start.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Chris Miller]

You can find a list of all files of the project, when you just USE the pjx file, it's a table.

But also with this list, you can't be sure whether some file is used indirectly, as Mike indicated with a macro substitution the build process would not know what is in a variable that's used in macro substitution and so it can't know which file could be used within that. Anyway, even in the most straight forward (usual) way of using excluded files - reports - a setup project or a distribution would need to provide these files separately, so you should know from that. The project surely also had a setup for it done with what is provided with VFP, the Installshield Express version or - also popular - an Inno setup. That would tell you what files other than an EXE or DLL built from the VFP project(s) are distributed alongside, apart from the runtime DLLs you always also need anyway.

So I'd look for an Installshield Express or Inno Setup project that tells you which files other than those includde in the PJX table(s) are necessary.

Well, and even then, more files might just be for the developer himself, like documentation.

The entirely other point of view is, that an EXE would always not just embed all files just because they are in the project folder, so you won't get a smaller executable just because you remove files. And there's the feature of files within a project that are marked as excluded, so they don't get embedded into the final execdutable. It's a default of DBFs to be excluded, for example, the table files, as you usually will finally use these tables separate from the EXE, within an EXE you can embed DBFs, but then they are reaonly and you could only store some static data in them, as you can't modify such embedded DBFs. Such DBFs obviously are still used and not obsolete.

Overall, I only see a waste of time in getting rid of files, unless you talk of gigabytes of log files that you don't need to keep. Otherwise, you only risk something important to be erased.

Just in comparison of how I would work, important would not just be all the files that are items of a project, so you can't use the simple approach of keeping everything listed in the PJX table and delete the rest as unimportant. If I were you I would let this go on organically, i.e. look into what's there to understand what it is and whether it's obsolte, so let the project grow (or shrink in this case) organically, not in a one time sweep where you don't yet have a grip on the project. If the number of files overwhelm you, you should perhaps find ways or use ways you use in other programming languages project to become familiar with it. To me that's usually single stepping through code, i.e. using the debugger.

Chriss

 

[Mike Lewis]

This method is far from perfect - mainly because it only shows the file list in a modal window; you probably want to output it to a text file or to print it, which you cannot do in this case.

It occurs to me that there is an easy way round that. With the project open in the development environment, do this in the command window:

USE (_vfp.ActiveProject.Name) AGAIN IN 0 ALIAS MyProj NOUPDATE
SELECT Name, Type FROM MyProj INTO CURSOR csrFiles

This will give you a cursor with all the file names (in the Name column) along with a one-letter indication of the file type (for example, P = PRG, V = class library, K = form, R = report).

You can then browse the cursor, feed it into a report, etc.

This still doesn't solve the problem of indirect references (macro substitution, etc.). Also, it's not really what you want. You presumably want a list of the files that are NOT in the project, so that you can clean them up. But it's a good start.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Chris Miller]

That's Basically what I also said, as _vfp.ActiveProject.Name is the filename of the project you have open. Whether you do the SQL Select or just BROWSE the PJX file, is up to you. If you browse the file you'll see the name field is a Memo which just shows as the word Memo in the browse window, so SQL Select turns it into a shorter char field, which displays directly. Instead, you can double click on a memo, see the value in a popup, keep this popup window open and scrolling through the records you see the memo values in that popup window, too. Also, you can see the field "Exclude", which tells whether a file in the project is excluded from building it into the final EXE in the build process.

As Mike said, this all does not tell anything about the files that are not even items of the project (or records of the PJX) but you can clearly easily get a simple list instead of a visual overvew in the "All" tab in a treeview. The tree has the advantage to sort the items by types, whereas the PJX records are just the chronological order of adding items. Every record is about the (main) file of a project item and no matter in which order, a list is better suited for you, of course.

One caution warning: For tables the PJX will only reference the main (dbf) file, other files with same name but extensions fpt and cdx also belong to the table, VFP tables can have one, two, or three files, depending on field types and indexing, plus there can be secondary index files, which can even have a different stem name. Tha same is true for forms, reports, class libraries. So from that perspective the file list in the PJX is not complete, it only lists the major files of databases, tables, forms, classlibraries, reports, etc. Rule of thumb is, that all files with the same stem name and different file extensions belong together as file groups, most of the time file pairs like DBF/FPT, PJX/PJT, VCX/VCT, FRX/FRT, etc. So half the files not directly in the records are still important and not junk.

Chriss

 

[Vince Wen]

Thank you Mike and Chris.

Chriss, it uses Inno setup. And I know the exact files together with the EXE are needed for the distribution. What troubles me is the 6GB of files/folders currently there to compile the EXE. A lot of that, I suspect, are backups and misnamed files. I am sure there are also files used for failed features or earlier features that was removed, but the files are still remaining. I am taking a fairly low risk approach. But I would still prefer to reduce this to something more reasonable to put in git to start.

Mike, thanks for the help. I'm not sure if it uses indirect references (macro substitutions) at this stage. But I think your approach will at least give me a sense of the folder/file structure, and I can start by getting rid of the backup folders and erroneous files.

I would love to get a 'clean' list of files that are used and referenced. But perhaps for now, I should be content with just getting rid of the clear garbage and reducing this to something more manageable size.

 

[Chris Miller]

Sounds good to me, and good luck.

Chriss

 

[Vince Wen]

I know I am supposed to start a new thread. But while I have you - part of the update process for the software is to manually and laboriously change a list of variable names and filenames each year (the software requires 'rolling over' the year in variables and filenames from '2022' to '2023' to '2024', etc.)

I was thinking about automating this with powershell script. Is this the predominant way to automate this amongst VFP9 developers?

 

[Chris Miller]

Why does this need a "roll over" in the source code? You have YEAR(DATE()) to give you the current year and so these year numbers can be calculated and then turned into a string for filenames, etc.

Chriss

 

[Chris Miller]

More generally speaking, why would I use Powershell to modify something that's VFP - VFP is a general purpose language that can, of course, do that itself, even if it would come down to generating code per year or per whatever "dimension".

Chriss

 

[Mike Lewis]

I was thinking about automating this with powershell script. Is this the predominant way to automate this amongst VFP9 developers?

Short answer: No. The predominant way to automate most things amongst VFP9 developers is to use VFP.

More to the point, any software that requires the programmer to change something in the program every year sounds like a very bad design. I feel sure there must be a better way of going about it.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Vince Wen]

In short, it is bad design.

This is an accounting software, and each new year requires us to add the latest year to it. However, this code has been passed from developer to developer over the years such that the way to "roll over" a new year is to manually add/edit variable names and add/edit filenames. I prefer not getting into why it has become this way...

The way to do it logically would require completely refactoring the code. In which case, you may as well rewrite it in a different language.

 

[Chris Miller]

The way to do it logically would require completely refactoring the code.

I don't see what you see, but imagining a lot of code with the year number directly in it, not as a variable, of course that would require refactoring all these year numbers given as literal numbers be replaced with expressions that automatically adjust to the current (or next or previous) year.

But that's not a big deal to do once and then have it fixed forever.

Chriss

 

[Chris Miller]

Notice, there is one rather weird function called SYS() in VFP, that is not a simple function but a collection of functions of which you could say, they didn't made it into the final product with a normal name, but just a number. And some of those are quite frequently used in alll code. If I imagine your current way of changing code from year to year with a search & replace, then you'd also have changed the call of SYS(2023) to SYS(2024), the former is giving the %TEMP% folder name, the latter is telling whether the last report run terminated early and SYS(2025) is undocumented.

So the search&replace approach isn't working very well with just the numbers.

Otherwise there is project wide search & replace in the VFP IDE in the form of the "tool" (in the Toools menu) if Code References. So that change can be done - with more caution than just replacing 2024 with 2025 - without any code, just with a search&replace. I'd recommend to only use the search feature (without replace) to list the places in the code that need to be using expressions giving the year number necessary.

Chriss

 

[Mike Lewis]

It may or may not be the case that it would require refactoring. I'm not in a position to judge that. But I don't see why it should require the whole thing to be rewritten in another language.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Chris Miller]

By the way, to get a key number of the amount of code you could do what Mike recently recommended in thread184-1829742 and use the Documenting wizard to get a listing of all code.
Edit: Not for refactoring all the year numbers in one place (you can't reverse the process and get everything in its right place) but to get an estimate of how much there would be to rewrite. I agree with Mike this is not necessary the simpler thing to do, not even ironically/sarcastic.

I basically see three major scenarios:

1. You have a small code base: The qestion then is, isn't there a standard accounting software package that can replace this whole project, no matter how customized it is for the company?
2. You have a huge code base, then it surely isn't easy to just rewrite this, no matter how good reasons you have, starting with lack of VFP knowledge. It's something you need to acquire to rewrite it anyway.
3. Yoiu have a mid size code base. Besides the question what is mid size it could still be replaced by a standard software or be too much already to rewrite it from scratch without any knoowledge of the project. There's really just a thin line where a rewrite is feasible. It would get more promising, if you already have an accounting software in your own portfolio that you can customize to the needs, ideally not by needing to dig into the code but asking for the specifics from the users, but that can also take a long time and be prone to errors.

Chriss

 

[Mike Lewis]

So, for now let's accept that you are stuck with this awful design. And that you haven't got the time, resources or expertise to refactor or rewrite. You need to automate the annual change.

It's difficult to suggest a firm solution without knowing more about the code. But let's assume that you have something like this:

[tt]USE Sales_2024 IN 0 ALIAS sales[/tt]

and you need to change it to this scattered around your code:

[tt]USE Sales_2025 IN 0 ALIAS sales[/tt]

And that you have many other changes to make of a similar type.

Start by identifying every instance of the code that needs changing. This is a one-off task, and is the same as you would need to do in any case every year if doing it manually. In each case, note the name of the PRG and the actual code that needs changing.

Then write a VFP program that takes in turn each of the relevant PRGs and reads it into a variable. Use FILETOSTR() for that. Then use STRTRAN() to find and replace the year numbers in each of the relevant lines of code. So something like this:

lcPRG = FILETOSTR("SalesProg.PRG")

lcOldYear = "2024"
lcNewYear = "2025"

lcOldCode = "USE Sales_" + lcOldYear + " IN 0 ALIAS sales"
lcNewCode = "USE Sales_" + lcNewYear + " IN 0 ALIAS sales"

lcPrg = STRTRAN(lcPrg, lcOldCode, lcNewCode)

* Repeat the above for each of the relevant lines of code.

* Then write the code back to the file
STTOFILE(lcPrg, "SalesProg.PRG")

* Repeat for each program

Clearly writing this program will involve some effort, but it will be a one-off task. Once you have done it, you can just run the program every year, only needing to change the values of lcOldYear and lcNewYer.

Note that the above is not production code. It is only meant to give you the general idea. You will of course need to attend to taking suitable backups, and also recompiling the programs and rebuilding the project.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Vince Wen]

Hi Mike, Thank you. Yes, it is more or less what you laid out. It also involves inserting the new year in various CASE statements and creating new files (for each new year) and adding new fields to .dbf (not sure if MODIFY STRUCTURE is something that can be automated). Regardless, it sounds like you would do this in VFP rather than powershell script. Thanks for laying out the sample code in some detail.

Hi Chriss,
Thanks for the heads up regarding "SYS()". I would never have known. But in short, the current update involves search&replace of the year (as you guessed correctly). Also, as mentioned to Mike above, inserting some new code in various CASE statements (for the new year), adding new fields (for the new year) to .dbf and creating new files (for the new year), etc.
To give some context, this is an accounting software we sell into a very specialized industry niche. Ie. this isn't used by us internally. It is a product we sell, so it isn't really a matter of replacing it. The choices for us are: (1) writing some script to do the annual updating (which was the original intent of this thread) (2) refactoring the code to make it long-term maintainable (as you/Mike recommend) (3) rewriting the code in a different language or (4) a combination of (1) (2) & (3). In fact, (2) and (3) don't conflict with each other, as we can continue to have VFP app as a desktop app and have it also rewritten as a browser app to give customers a choice. This starts to become more of a business decision though, if you know what I mean.

 

[Mike Lewis]

not sure if MODIFY STRUCTURE is something that can be automated)

Yes it can. Use ALTER TABLE for that.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[tomk3]

Hello,


I like Mikes answer writing a little program which does the changes.
For search / replace I suggest to have a look on "GOFISH"

https://github.com/mattslay/GoFish

https://github.com/VFPX/GoFish

Best regards
tom

Btw : Nearly everything can be done in VFP if needed with add. libs like chilkat, vfpencryption, ... or addons like foxypreviewer
With the exception of device drivers and action games (not sure on that )