Limitations of the Locate command 3

[Jerim65]

My application may well be used with files on a NAS which are returned in UNC format.

I have a table of previously used paths, some on a NAS.

When I test the selection of a new path gained with getdir()

I have tried both a SEEK on an indexed field (which was not possible because the path could be too long for an index ) and LOCATE

if the selected path is, say

\\NASSERVER209\MYUSERDATA\PROJECTS\__ABC PROJECTS - MINE\MY PROJECT 2010\WETTON_REAL 2010 V7_TESTER\

the code below fails when the path is already in the mypaths table.

Select mypaths

Locate For mypath = mynewpath

If !Found()

	Append Blank

	Replace mypath With mynewpath

Endif

So I have tried

Select mypaths

Scan

	oldpath = ALLTRIM(mypath)

	If !oldpath = mynewpath

		Loop

	Else

		doit = .T.

	Endif

Endscan

If doit = .T.


	Append Blank

	Replace mypath With mynewpath

Endif

Another routine in my application is doing the exact same thing with a LOCATE and this is failing as well.

So if my users keep their files on a NAS with a deep folder structure my code is failing.

The code works in both cases with a local drive path.

Can anyone advise me what to do please.

Coldan

 

[JRB-Bldr]

Let's start with your first statement...

"which was not possible because the path could be too long for an index"

The Index Expression and its TAG do not use the Path.
So I don't understand what you are saying.

   * --- Create the Indicies ---
   cMyDBF = <Pathed Data Table Name>
   USE (cMyDBF) IN 0 ALIAS MyTable EXCLUSIVE 
   SELECT MyTable
   INDEX ON Field1 TAG Fld1
   INDEX ON Field2 + Field3 TAG 2Flds
   <etc.>
   USE

   * --- Now somewhere else in the program ---
   cMyDBF = <Pathed Data Table Name>
   USE (cMyDBF) IN 0 ALIAS MyTable
   SELECT MyTable
   SET ORDER TO 2Flds  && Activate desired Index

   * --- Perform the Seek ---
   cSeekKey = "ABCD" + "XYZ"
   SELECT MyTable
   IF SEEK(cSeekKey)
      * --- Seek Expression Found ---
      <do whatever>
   ENDIF

Now were is there a Path in the Index or in the Seek?

Let's eliminate that issue before we look at the other.

One other thing to look at will be the setting for SET EXACT
If it is ON, then what you are looking for (SEEK or LOCATE) must be an EXACT match.

Another thing will be the length of the data strings used in the SEEK or LOCATE. Are they exactly the same length as the fields being searched for?

Good Luck,
JRB-Bldr

 

[Jerim65]

Hi JRB-Bldr

Sorry to confuse.

I am trying to compare strings.

One string is a path gained through a getdir() and the other is a field in mypaths.

The first is compared with the second - the list of paths in the field in mypaths table.

If both strings are the same length, SET EXACT has no effect.

My strings are the same length hence I asked if there was a limitation OR how can I make the comparison.

The rouitne in my app is dead in the water if I cant get the comparioson to work.

I have a beta user screaming about this already.

Thanks

Coldan



Coldan.

 

[GriffMG]

I have had a similar problem, and my answer is not going to be directly related to your question - but might get you going.

I have an application that backs up files from a remote location to a server here. The machine here send a list of the files it has already (with times and dates) to the remote machine, which then compares the list with what it already has - then sending back any that have changed or are new and what not. We are talking about about a couple of million files at each location, so LOCATE was no good to me - far too slow. As you say, you can't index on the full path either - the index has a key length limitation.

Good news though, index on as much of the key as you can, first 200 characters will do - most of the paths will actually be shorter than that.

Then you only have to process the records that match on the first 200 characters - using a regular do while... skip... enddo to locate or otherwise your path.

Good luck

Regards

Griff
Keep [Smile]ing

There are 10 kinds of people in the world, those who understand binary and those who don't.

 

[Olaf Doschke]

Jerim65,

240 chars is a genereal limitation of indexes, true. The sample path you gave is 100 chars long, not an example of exceeding that limit.

To answer the original question, LOCATE does not have that limit, but if an index exists on a field the Locate is optimised by that index and thus will have the same limitation of comparing just 240 chars. Remove the index and the locate will just compare the full values, eg of a memo field in the table you locate in with the variable value.

You may have mismatch because of Upper/Lower Case you may have mismatch because the field you use to store paths is c(254) and paths are longer. Actually a limit of path length is 260 unfortunately longer than the max char field length.

See here:

http://msdn.microsoft.com/en-us/library/aa365247.aspx

in the paragraph "Maximum Path Length Limitation": In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length for a path is MAX_PATH, which is defined as 260 characters.

The technical limitation of NTFS actually is about 32000 chars, but GETDIR() or GETFILE() will never work with such long paths anyway, even if you use flags making GetDir/Getfile use the windwos system dialog. Nevertheless be prepared paths may be longer than a char field can hold.

As the example you gave is still beneath all these limitations I assume you have another problem, though.

Bye, Olaf.

 

[JRB-Bldr]

I agree with the suggestions above.

In order to speed up the comparison, use a SEEK and if successful, then do a LOCATE on the next few records for an Exact match.

  USE MyTable EXCLUSIVE
  SELECT MyTable
  INDEX ON LEFT(UPPER(PathedFile),240)) TAG Close
  USE

  * --- Then elsewhere in your code ---
  cSeekKey = UPPER(LEFT(cChkString,240))

  USE MyTable IN 0 ORDER Close
  SELECT MyTable
  IF SEEK(cSeekKey)
    * --- NEAR Match Found, Now Check For EXACT ---
    LOCATE NEXT 5 FOR ALLTRIM(UPPER(cChkString)) == ALLTRIM(UPPER(PathedFile))
    IF FOUND()
       * --- EXACT Match Found ---
       <do whatever>
    ELSE
       * --- No EXACT Match Found ---
       <do whatever>
    ENDIF
  ELSE
     * --- No NEAR Match Found ---
     <do whatever>
  ENDIF  && IF SEEK(cSeekKey)

Good Luck,
JRB-Bldr

 

[JRB-Bldr]

I missed something in my previous post.

Since the Index Expression is built on 240 characters, the SEEK expression MUST be the same length to ensure success when there is a match.

So instead of:
cSeekKey = UPPER(LEFT(cChkString,240))
it should have been something like:
cSeekKey = PADR(UPPER(ALLTRIM(cChkString)),240)

In that way, if cChkString is longer than 240 characters, it will only do the SEEK on the UPPERCASE of the leftmost 240 characters.

And if cChkString is less than 240 characters, it will, after UPPERCASEing it, pad it out to match the Index Expression length.

Good Luck,
JRB-Bldr

 

[Olaf Doschke]

With EXACT OFF the expression seeked does not need to have the length of the index expression. You can SEEK 'A' with NEAR ON and EXACT OFF and find the first record starting with A or the nearest letter, if there is no record with A. Just as an example.

If the index does not pad with spaces padding the seek value with spaces also can make seek NOT find the path. But with one thing the padding really is better: If you have a more special, longer directory in the table, then seeking a shorter path you find it in the list already, even though it isn't in the list. Eg You find "C:\" in the list of recent paths if there is any path of the C:\ drive already in it. With padding you only find a match of "C:\"+Space(237), if there is the pure C:\ path in the data already.

Bye, Olaf.

 

[Olaf Doschke]

Back to your orignal problem: You have a table mypaths with a field mypath containing unc paths. The user chooses a path with Getdir and you wnt to add it to mypaths only, if it is not in there.

Your initial code looks quite good

Select mypaths

Locate For mypath = mynewpath

If !Found()

    Append Blank

    Replace mypath With mynewpath

Endif

This would work if paths stored are upper case, getdir returns uppercase directories and mypath is a memo field.

To make sure I'd modify this:

Create Cursor mypaths (mypath M)

mynewpath = Upper(AddBS(GetDir(...)))
* optional
* mynewpath = uncpath(mynewpath)
Select mypaths
Locate For mypath == mynewpath

If !Found()
    Insert Into mypaths (mypath) Values (mynewpath)
Endif

Now you won't have a problem with long paths and capitalisation. And as you store into a memo field the values will be able to be longer than in a char field, will not be padded like in a char field and you can compare with ==, as you store the trimmed values into the memo field you can locate for the exact match. AddBS() is always adding a backlash to the path AND it does truncate the string.

The optional commented line calling a function UNCPath() points to another thread of yours, where you were shown how to find out the UNC path of a drive letter path, so that should be a function you could implement from the answers you got there, which should make any path a UNC path, if there is a drive mapping, only local paths will remain drive letter paths.

Bye, Olaf.

 

[Jerim65]

Olaf,

I have already placed a uncpath conversion in my app after getdir().


Regards

Coldan

 

[Olaf Doschke]

Good,

applying uncpath conversion is not the only difference in my version of your code, though. If you mean to say "thanks, I already tried that" and didn't try my version, then I suggest you do so.

Bye, Olaf.

 

[Olaf Doschke]

Now I've applied my code to some sample data, getting the expected result:

Create Cursor mypaths (mypath M)

Create Cursor inputdata (somepath M)
Insert into inputdata values ("\\NASSERVER209\MYUSERDATA\PROJECTS\__ABC PROJECTS - MINE ")
Insert into inputdata values ("\\NASSERVER209\MYUSERDATA\  ")
Insert into inputdata values ("\\NASSERVER209\MYUSERDATA\PROJECTS\__ABC PROJECTS - MINE\MY PROJECT 2010\WETTON_REAL 2010 V7_TESTER   ")
Insert into inputdata values ("\\nasserver209\myuserdata\projects\__abc projects - mine\my project 2010\wetton_real 2010 v7_tester\ ")

Scan
   mynewpath = Upper(AddBS(inputdata.somepath))
   Select mypaths
   Locate For mypath == mynewpath

   If !Found()
      Insert Into mypaths (mypath) Values (mynewpath)
   EndIf
EndScan

Select mypaths
Go Top
Browse Nowait

This is putting the first three paths into the mypaths cursor and rejecting the last one, as it's just the lower case version of the third path with a backlash.

All ingredients are important:
1. M = Memo field is storing variable length data, not padding them like char fields, can contain any length of path.
2. AddBS() is adding a backlash where it's missing and also truncates the path.
3. Upper() takes lower case letters to upper case, making values comparable. If you prefer lower case paths in the mypaths table you can also use the Lower() function.
4. == is making an exact match, so unlike VFPs normal string comparison with = and the default EXACT OFF, shorter paths don't get a Found() = .T., if a longer path is already in mypaths, only exactly the same paths are found as already existing in the table.

All these changes may be subtle, but all of them make it work.

Bye, Olaf.

 

[GriffMG]

Olaf,

Isn't that going to be a bit slow if there are a large number of records?

Regards

Griff
Keep [Smile]ing

There are 10 kinds of people in the world, those who understand binary and those who don't.

 

[Olaf Doschke]

Griff,

yes, this will be slow with thousands of paths, but it's working. As I think a table holding "previously used paths" for a "recent documents/paths" feature will only store a few paths, this would be sufficient.

Bye, Olaf.

 

[GriffMG]

Olaf,

I must have missed the bit about using it for a 'recent documents' approach - that should be fairly modest, and thus pretty quick.

B-)

Regards

Griff
Keep [Smile]ing

There are 10 kinds of people in the world, those who understand binary and those who don't.

 

[Olaf Doschke]

Well, "previously used paths" is a quote from the inital post, "recent documents/path" is my interpretation. Coldan (Jerim65) didn't actually talked about numbers.

You can take JRB-Bldr's approach with SEEK mixed with locate, I actually don't like the NEXT 5 in the LOCATE, but this would also work okay with just a bunch of paths. Still you need to make paths comparable with UPPER() or LOWER(), with AddBS() and/or truncation, if you want a reliable match, you need a normalized path you actually can compare.

Bye, Olaf.

 

[Jerim65]

Olaf,

You said

>applying uncpath conversion is not the only difference in my version of your code, though. If you mean to say "thanks, I already tried that" and didn't try my version, then I suggest you do so.>

No, I meant I had also modified my code with the other elements you mentioned but I already had the uncpath in another part of my code - the getdir() is inside seldirdlg() now - thank you for your very valuable and welcome help.

The same thanks to others that are contributing here also.

Coldan

 

[Jerim65]

Routines involving UNC paths now working satisfactorily.

Thanks everyone.

Coldan

 

[GriffMG]

Cool,

Good luck

Regards

Griff
Keep [Smile]ing

There are 10 kinds of people in the world, those who understand binary and those who don't.