When & WhenNot to use &var and (var)

[stanlyn]

Hi,

Can someone explain the proper usage of macto expansion such as &var and evaluation such as (var)?

I've been using them for years without really knowing exactly when they should be used and/or avoided. When I use them and I have an error, I simple switched to the other version, and all without really knowing the pros, cons, and gotchas. Typically I use:

1. lcVariable
2. &lcVariable
3. (lcVariable)

And yes, I've read the help docs, but asking here more about real life experiences with them. Like a rule of thumb...

Thanks,
Stanley

 

[Tore Bleken]

The easy answer is to NEVER use macro, period. There are always other and much better ways, specifically (lcVar), Evaluate(lcVar), TextMerge() and ExecScript(lcVar). When working with objects, I find using Store makes much more sense than using macros. I really can't think of any case where you will need a macro.

 

[Olaf Doschke]

Well, I thought I made it clear, that name expressions are usable anywhere but only needed in commands, all VFP commands, expect ! or RUN. Thee are synonyms for a foxpro command, but a foxpro command to call cmd.exe or command.com and/or any EXE via /N parameter and the rest of the command is external, the parameter part of the command is not using name expressions, you can't pass VFP variables, only their values. Macrosubstitution in this case is the only way to pass in variable values, as it is done before the code is executed, name expressions are not preprocessed, that's their big advantage performance wise.

More obvious, you can't use any VFP variable in VBA code, but of course composing VBA code in a string you can use macro substitution, the final code passed to the Office automation server. But don't confuse running VBA scripts with working in the OLE Object model. And the only other thing forwarding variables is SQLEXEC or SQL Passthrough in general, but that's VFP side processing of the commands sent, the remote backend is not seeing ?m.variable inside the passed in query.

Makro substitution also comes in handy in dynamic SQL, Tore might just use statements allowing parameterizations but no dynamic composition of SQL, I'd say that's a pitty, as you can compose SQL, whcih still is sql injection proof but easier to maintain than N prepared statements.

Overall, you don't get around having the detail knowledge what causes what and where its caused in detail or you're always surprised by what's happening against your expectations. It's always a chance to learn, don't tend to interpret your misunderstanding as a bug, that'll just make you use workarounds not needed.

Bye, Olaf.

 

[Scott24x7]

Stanlyn,
To understand all these "methods" (meaning, ways to accomplish things), you need to understand a bit more about the way things work also.
Variables are either a) passed by value or b) passed by reference. These various means of "macro expansion" as you mention may or may not be necessary depending on the scenario.
There are many who will shun the use of & (Macro Substitution) particularly in the VFP space, but the reason it is prevalent here has it's roots in FoxPro before it was an object oriented language (and was and still is an event drive, interpreted language). Event driven is the "forerunner" of object models and FoxPro was one of the languages available in this "before OOP" that took advantage of it. As a result, macro substitution became prevalent, and despite what others will say, it's not *that* horrible.

So let's take example 1 and 2 in your list.
1. lcVariable - this is a typical variable, store a value a reference to the value. Variables are just that -- references. So hence the, "pass by reference" phrase you may hear from time to time. Arrays are also references to variable sthat include a "vector" -- a means of specifying WHERE the variable is in a "list" (for simple terms). So to expand on this example your lcVariable could just as easily be laVariable (note that I'm using "lc" and "la" as convention for naming only, they have zero baring on the actual references or type of variable they store, there "l" helps the developer understand the intended scope of the variable "l = local", and 2nd character they type "c = character" and "a = array". But these are only for ease of reference in code, and can be changed unintentionally, so be careful).

So in a practical usage you may say something like lcVariable = mytable.myfield and what happens then is, a variable named lcVariable is created by VFP and whatever the contents of mytable.myfield are get ASSIGNED to that variable (where = is the "assign" operator). Memory is used to hold the contents. Let's pretend for a moment that the value is the word "OFF" (I have a reason for this later).

So now if you were to put in code that says MESSAGEBOX(lcVariable) VFP would return to you "OFF". This is passing the variable by REFERENCE to MESSAGEBOX. It's "by reference" because we give the function just the variable name, and it uses it. MESSAGEBOX() is a pretty clever function, because it also doesn't care what data type is passed to it (as a single variable), it will determine what it is and display it for you.

Another way is to pass by VALUE. Value is just what it sounds like. Instead of passing the variable name, you actually shove in the "value". So, when we consider the example so far, there is a VARIABLE called lcVariable. It's VALUE is "OFF" (which we assigned to it from a field in a table).

Now, the "Macro Substitution" part does just what the latter part sounds like as well "SUBSTITITES" (in this case, puts the VALUE there, not the reference to the value.

So interestingly then with MESSAGEBOX, if you called it this way: MESSAGEBOX(&lcVariable) you would get an error. What VFP "Sees" in this case is actually MESSAGEBOX(OFF) the "macro substitution" gets literally interpreted by VFP as what the value of the variable is. Now, you can see a new behavior if you were to issue MESSAGEBOX("&lcVariabl") you will again get a messagebox display that just says OFF. What VFP sees in this case is MESSAGEBOX("OFF") so it prints OFF as we would expect to see.

So in your examples 1 - Is typical variable 2 - Macro Substitution (noted by the & added to ANY variable name) and your 3rd example is NAME SUBSTITIUTION.

Name substitution actually works almost exactly like macro substitution, but has a couple of advantages 1) it's "faster" (we're talking milliseconds) and 2) it works with more complex expressions, but 3) doesn't work with every case that macro substitution would... there's some reasons for that.

So if you're working with a form, let's say you create an property for an object. So that object is now referenced by it's path. myobject.propery for example.
Now thinking back to variables again, and passing them by reference, or value (and there by substitution methods in many cases) not all substitutions work the same way.
Put into context. I have a variable in my form that holds the name of the primary table I use in that form. So ThisForm.ActiveTable I can assign with ThisForm.ActiveTable = MyTable
To make this reusable across every form I use, and I don't have to reprogram the table name every time, I then use that reference in my code (especially for things like navigation button).

So now instead of in code entering SELECT MyTable (which is a "hard coded table name") I can say instead SELECT (ThisForm.ActiveTable) -- note here, I am using NAME substitution, which is noted by the () that are not a Function. (If it were a function it would be SELECT(ThisForm.ActiveTable) but SELECT is a VFP function that puts the workarea on the focus table (by many means). BUT, I bring this up because & substition will NOT work in this case. Because it would then have to be SELECT &ThisForm.ActiveTable and & works on the VARIABLE name that follows, this is an OOP relationship instead. SO another substitution method is needed. This is because SELECT expects to see either the work area number (litteral) or workarea name (litteral) and is why you don't "" it. So for example, using MyTable as the table name, you could:

SELECT MyTable
SELECT (Thisform.ActiveTable) (where "MyTable" has already been assigned to the property -- VFP Sees SELECT MyTable - after the substitution takes place)

Now concerning EVALUATE() function, this works different still.
If you issue:

SELECT EVALUATE(ThisForm.ActiveTable) VFP will generate the error "Variable MyTable is not found."

Which is an interesting clue about what's happening here... because EVALUATE is trying to pass a REFERENCE to SELECT and NOT a VALUE.

So NAME substitution works by passing the VALUE, EVALUATE returns the REFERENCE which works with functions that pass/share information by reference. So each of these have a different purpose, and are used differently. If we assigned lcVariabe = "MyTable" then:

SELECT (lcVariable)
or
SELECT &lcVariable

provide the same result, but the argument is that () Name substitution is faster than & Macro substitution.
There are times when name substitution doesn't work, and you need a different method. In these cases macro substitution may be necessary, EVALUATE is not an automatic "fix" to & (macro sub) nor is Name substitution. In many cases the "variable" will do fine, but that's a time where the function/call/command is expecting to receive a variable reference instead of a literal.

If you inherit code, or have to deal with backward compatibility, & is your best friend. I do find it comical the number of fox developers who harp on the & usage, and then find them running multiple DO LOOPS in succession which adds enormous amounts of time onto the run time. IF you're doing everything else right, the usage of & will matter ONLY when you have extensive loop processing where the code gets called over and over. The key thing here is because VFP has to process it every time, whereas with NAME substitution, at run time, it manages it differently in memory, and as a result once the expression is "unpacked" it keeps that in memory. With Macro it unpacks every time... HOWEVER... there may be clever/tricky situations where you want this behavior. If the value utilized by & in the loop changes, then the only way you'll get an unpacked change is with macro sub.
Hope that helps.



Best Regards,
Scott
ATS, CDCE, CTIA, CTDC

"Everything should be made as simple as possible, and no simpler."

 

[Olaf Doschke]

>EVALUATE is trying to pass a REFERENCE to SELECT and NOT a VALUE
You got that wrong. Evaluate evaluates expressions, it's never usable for names, if you EVALUATE("variablename") it'll return the variable VALUE. If you EVALUATE("customer") that doesn't work, as customer is not a variable or expression, it's just a table name, you can only evaluate customer.name or such expressions returning a VALUE, so in the end EVALUATE evaluates expressions to their VALUE and never references.

Bye, Olaf.

 

[Scott24x7]

Ok and I see that now. But the point is if you EVALUATE(ThisForm.ActiveDBF) who's VALUE is the table name, then it passes only that back to SELECT and it doesn't work as a substitution method in this case. (Which was my key point).

Best Regards,
Scott
ATS, CDCE, CTIA, CTDC

"Everything should be made as simple as possible, and no simpler."

 

[Tore Bleken]

Olaf,

why use Run when you can ShellExecute? Still no need for macros. Show me an example where you must use macro, and I will (try to) come up with an alternative.

And I also build a lot of dynamic SQL expressions, and run them with ExecScript(). When I wrote that you NEVER need to use a macro, I really meant it. I searched all my recent code for "&" and all I found was "&&" for my inline comments.

And regarding Evaluate(), it's far, far more powerful than you clearly have discovered, it can even run code! (And no, I can't give you any sample on top my head, unfortunately I am too busy preparing for the SWFox conference in Arizona next week).

 

[Mike Lewis]

This is partly a matter of personal preference. Personally, I disagree with Tore when he says "The easy answer is to NEVER use macro, period." I generally prefer to use name expressions (expressions enclosed in parens) where possible, in particular when the command requires a literal file name or alias name (such as[tt] USE TableName[/tt]). But there are times when a macro can be useful.

For example, if you need a particularly complex SQL SELECT command, where you don't know exactly which clauses you will need until run time, it is often easier to buld the command step by step, storing it in a variable, then macro-expanding that variable. I don't see any problem in doing that.

But, as I say, it's partly a matter of personal preference.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[Olaf Doschke]

Tore, it's not news you can run code with evaluate, expression can of course contain function calls.

You can prepare sql and finally EXECSCRIPT it, yes, but that has its separate variable scope, so whenever you use variables (parameters) in your statement, you pass it in? Or do you use public or private variables? Or do you embed them? Then how about injections in that case? Never trust user input, what about that? If a where clause like NAME=m.lcName is used and m.lcName comes from THISFOM.txtUsername.Value, then you can neither use m.lcName nor THISFORM... inside EXECSCRIPT. EXECSCRIPT doesn't save you much, it also compiles, it even writes out a tmp file it compiles, it can only be faster, if you really have a script, several commands, that can't be done by macro substitution. So your example of using it for SQL is weak even just in the performance aspect.

What's true is, that you can avoid macro substition in most any case and doing so is a good way to getting used to not having it in eg .NET.

Bye, Olaf.

 

[Scott24x7]

These are all good points. And I agree with Mike on this one too, I would describe in my case "personal preference" and more a matter of "conditioning". Readability for me of macro sub is so easy and automatic, versus name substitution, where the difference is a space between the (). I see () everywhere. But a quick glance and seeing an & can only mean 1 thing. I don't "prefer it", I process it differently/faster and I think its far more obvious when seen. That comes from decades of FPD and FPW.

I'm VERY conscious of their usage in the case where any kind of looping expression is used. Which is really where this "speed" issue matters. In the typical "user forms" it's not so dire that a & is used to select some table, or update the SET ORDER TO command...

And years ago I remember building dynamic search & query for users where they put in various parameters in a screen, and that created SQL call, and as described a macro sub in these cases is not the end of the world.

Olaf, I'm curious about your mentioning of macro sub and .NET though. How are these related?


Best Regards,
Scott
ATS, CDCE, CTIA, CTDC

"Everything should be made as simple as possible, and no simpler."

 

[Tore Bleken]

Olaf,

with Execscript I use private variables and pass them with ?pcVariable. I have not noticed any speed degradation.

Please pardon me, but I won't be able to send any further comments for a while, I have to give other matters top priority.

 

[Olaf Doschke]

That's fine, you can do your own experiments to see what I mean, when you have the time. Starting with EXECSCRIPT("? ADir(laDummy,Addbs(GetEnv('TEMP'))+Program()+'.fxp')"), you'll see it writes out a file, something macro substitution does not. It'll be cached by OS, but still is an offset of performance you don't have via macro substitution. With SQL it won't matter much against typical query durations, though some queries can be very fast, if they are simple. In that case it's an offset you won't want to have.

Bye, Olaf.

Also try these to see:
[pre]EXECSCRIPT("? ADir(laDummy,Addbs(GetEnv('TEMP'))+Program()+'.*')") && will show 2, a tmp and an fxp file are generated.
EXECSCRIPT("? FileToStr(Addbs(GetEnv('TEMP'))+Program()+'.fxp')") && will show the compiled code (you'll see TEMP inside it or any string literal, you'll also see both file names. Normal for any FXP.
EXECSCRIPT("? FileToStr(Addbs(GetEnv('TEMP'))+Program()+'.tmp')") && will error 'File access is denied', VFP writes out the tmp file with exclusive access to it.[/pre]

 

[Olaf Doschke]

>curious about your mentioning of macro sub and .NET though. How are these related?
Well, .NET does not have macro substitution.

I'm saying, if you program in a way not needing macro substitution you can more likely translate that code to .NET, if macro substitution is a major tool for you, you'll have difficulties transferring that to .NET code.
Macro substitution was one major thing standing in the way of porting VFP to be part of the .NET world, so it was continued as separate VFP7 product instead of being an Visual Studio included language, as it was up to VFP6.

Bye, Olaf.

 

[Tore Bleken]

Olaf,

to show you the power of Evaluate, I created this small program. Actually I was amazed myself, maybe I can reduce my use of ExecScript.

[pre]private pcLast
pcLast='bleken'
lcX="sqlexec(goapp.nDataHandle,'Select * from customerlist where lastname = ?pcLast')"
evaluate(lcX)
Browse normal
[/pre]

 

[Olaf Doschke]

What's the surprise?

And why would you evaluate this, if you can do [tt]sqlexec(goapp.nDataHandle,'Select * from customerlist where lastname = ?pcLast')[/tt] without first putting it into EVALUATE()? Individual SQL code would also not need this, as it would go as second parameter of the SQLEXEC(). In this case there even is no need for a private variable, evaluate runs in the current context.

Bye, Olaf.

 

[Tore Bleken]

This was just a very simple example. My point is that evaluate(lcX) behaves just like &lcX or ExecScript(lcX), in other words, you can use Evaluate() to run code.

 

[Dan Freeman]

This is partly a matter of personal preference. Personally, I disagree with Tore when he says "The easy answer is to NEVER use macro, period."

I agree with both of you a little bit.

A lot of it boils down to style, but there's also a code quality issue. (Which is also hugely reflective of style.) I've had recent instances where I removed 300+ lines of code (in a 12-CASE DO CASE block) and replaced it with about 10 lines in a FOR loop, using &macros to declare variables where the names were created based on the current date. (The entire enterprise was silly and should be retired anyway.) Code readability is a little questionable, but more importantly now all of the code runs every month rather than having one of the CASEs running each month. (The most difficult code to debug is the code that only runs once/year!)

Never say never, but do things knowing all of the ramifications.

 

[Mike Lewis]

Yes, good point, Dan. It was the capitalised NEVER that I was dubious about.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

Visual FoxPro articles, tips and downloads

 

[stanlyn]

Thank you all so much for this detailed discussion. I never expected this level of input, on clarification that I probable should have asked years ago.

Like I first stated in my opening question, I've been using all this for years and when I ran into a situation where something didn't work, I simple switched to another way of doing it until it worked, and moved on. I've always had the questions and have looked for definitive answers, and finally yesterday I asked... I guess its never too late.

Scott, where did you get all the detailed info on how they all work "under the covers" as I haven't seen any docs as detailed as yours, and thank you for that. Your section and this whole thread will be heavily studied the next time I get into a non-working situation, to understand why it won't work.

Olaf, thanks for the vfp to .net tidbit and your examples, as I can use that now elsewhere in this project.

Again, Thanks
Stanley

 

[Scott24x7]

Stanlyn,
Some of it comes from careful observation of how this works over a very LONG period of time. (Back in "the day" before I went into developer retirement, I was the lead for database and application development for Indiana University... we had a lot of access to people). Some of it came from attending developer conferences (particularly the first VFP DevCon in Phoenix where VFP 3 was released) and there were a TON of Microsoft developers available to discuss how things worked. (Just for clarity, I don't mean the first ever DevCon but the first DevCon when VFP 3 was released, in 1996). And then again the following year in 1997 in San Diego, there were tons of Microsoft developers from the VFP program on panel as well as freely available, if you could grab them (which I did). So some of this, I've never seen documented, it's just information that the MS development team provided at the time. Some also probably came from FoxPro Adviser, as I was a long time subscriber to that magazine as well, and have a lot of the still on CD (though they are in storage with the rest of my Fox Dev documentation). I'll be able to retrieve them in November (but they are about 200+ miles from me at the moment, in my storage unit, which is unfortunately inconvenient, but living in Tokyo, it's kind of unreasonable to store it long term a block or two away. I'm making a trip out in Nov, and plan to bring it all back with me then.) I prefer to have citable references, but for now, it's just all from memory. (Which explains my errant representation of return values of EVALUATE() but then again, I almost never use it.)

Glad it helped.

Best Regards,
Scott
ATS, CDCE, CTIA, CTDC

"Everything should be made as simple as possible, and no simpler."