STRTRAN() performance 2

[mkrausnick]

I wrote a routine that filters names, removing garbage characters. For the first crack at it, I just wrote a series of STRTRAN()'s, one per character I want to remove. The problem is that it's waaaay too slow. Only a few of the names have garbage characters, and when I'm processing 10's or 100's of thousands at a time, it just takes too long.

Would it be faster if I use this construction?:

if '*' $ lcName 
    lcName = strtran(lcName,'*','')
endif

rather than unconditionally doing the strtran()? Or is there a faster technique?

Thanks for your help.


Mike Krausnick
Dublin, California

 

[Mike Lewis]

Mike K,

I agree with Mike Y. A CHRTRAN() will usulaly be faster than a series of STRTRAN()s. In fact, I often use a nested CHRTRAN() to remove garbage characters, like so:

lcGoodChars = "ABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789"
lcStr = ;
 CHRTRAN(lcStr, CHRTRAN(lcStr, lcGoodChars, ""), "")

The advantage is that you don't have to know in advance what the garbage characters are. You just need to know the good ones.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

My Visual FoxPro site: www.ml-consult.co.uk

 

[mkrausnick]

Wow Mike Lewis, that's very clever! I'm using that code!

Thanks to you both.

Think this thread has enough Mikes?



Mike Krausnick
Dublin, California

 

[Olaf Doschke]

SET LIBRARY TO FOXTOOLS
? StrFilter("****Mr. BIG STARR****","abcdefghijklmnopqrstuvwxyz ABCDEFGHIJKLMNOPQRSTUVWXYZ.")

 

[steve4king]

That looks like it could be pretty useful.
Please forgive my ignorance, but how would one apply it?

Would this be used to capture invalid characters during an insert? Or would you implement this to search through a database to find invalid characters.

Mind throwing in a little context code or descriptions for the curious onlookers such as myself?

Thanks
-Steve W.

 

[Olaf Doschke]

Well, you may find improper chars within a database. simply use it like this for example:

USE sometable
BROWSE FOR NOT StrFilter(sometable.somefield,listofGoodchars)==sometable.somefield

Just be sure to not forget space in the list of good chars.

And vfp also has ISDIGIT() and ISALPHA() to decide which chars are good (you may not have had some foreign language letters in mind, like French, Spanish, Polish or German letters, ISALPHA() knows them, so you may simply create the list of characters by

Local lcGoodchars 
lcGoodchars = ""
for i = 0 to 255
  if isalpha(chr(i)) or isdigit(chr(i)) 
     lcGoodchars = lcGoodchars + chr(i)
  endif
endfor i
lcGoodchars = lcGoodchars + " .-"
? lcGoodchars

The case decides what is good chars of course. You may NOT want to allow digits in cases, or ONLY allow them. Whatever.

You could also do THIS.VALUE = StrFilter(THIS.VALUE,...) in InteractiveChange() events, but that may not work well with cursor positioning, eg the cursor may move as you insert an invalid char in the middle of a textbox. So it may be better to filter in the Valid() event of controls. You may try for yourself.

Bye, Olaf.

 

[Mike Lewis]

Steve,

This is a common thing to do in search routines, especially if you want a measure of "fuzzy" matching.

For example, if you want to let the user search on company name, you could store a version of the name that contains letters and digits only. You can then do the same with the seach term. That way, a search for MLC-Ltd would correctly find M.L.C. Ltd., for example.

Mike

__________________________________
Mike Lewis (Edinburgh, Scotland)

My Visual FoxPro site: www.ml-consult.co.uk

 

[steve4king]

Very cool Mike Lewis! I did not know of a way previously to search like that.. Gunna have to play with this one.