Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

Register Log in
  • Congratulations sizbut on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Stupidest Sort Problem Ever

Status
Not open for further replies.
careerquiz

careerquiz

Technical User
Nov 20, 2007
4
US
Hi everybody,

I have a really stupid sort problem that I haven't been able to figure out. Although, not entirely a novice I've just spent 4 hours trying to sort an array.

Okay here it is:

I have a database.txt file with a header row such as

name|address|category
John|122 Mockingbird|Tools

Okay, I'm using a small script to search the file and pull out matching terms and then place them in a template.

If I sort the using a sort such as

@lines = sort { $a cmp $b } @lines;

the script hangs for awhile then crashes. If I use a reverse sort such as

@lines = sort { $b cmp $a } @lines;

the script completes but the results are in decending order. This is driving me nuts. I'm sure it's something stupid. I think it may have something to do with the header row but not sure. Could really use someones help on this one. I've attached the script to this message. Thank you for your time.

 
Sort by date Sort by votes
Code: 
@lines=sort(@lines); #doesn't provide the list you need?

You need to find out why it crashes, but in the meantime, if you're getting the reverse sort working, then just reverse the list in the short term.

Would a hash be a better solution to this, assuming you won't have duplicated entries?

Paul
------------------------------------
Spend an hour a week on CPAN, helps cure all known programming ailments ;-)
 
Upvote 0 Downvote
I'm not going to look closely at a script of more than a few lines that is written/compiled without the strict pragma. One small typo in your script could be the source of problems.

There is certainly nothing wrong with the sort code you posted so the problem is elsewhere. Does the code that does the actual sorting have warnings (or -w) enabled? Could be your script is producing profuse amounts of warnings which can cause a script to hang and crash. Check the server error logs if you can.

------------------------------------------
- Kevin, perl coder unexceptional! [wiggle]
 
Upvote 0 Downvote
I don't see a call to 'sort' anywhere in the code you posted.

What's the content of @lines before you sort?
 
Upvote 0 Downvote
it doesn't have to. It will do the sort and then recreate the array with the new data.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[noevil]
Travis - Those who say it cannot be done are usually interrupted by someone else doing it; Give the wrong symptoms, get the wrong solutions;
 
Upvote 0 Downvote
I would say if you are having a problem trying shortening your script just to the sort part and the data causing the problem and post it here.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[noevil]
Travis - Those who say it cannot be done are usually interrupted by someone else doing it; Give the wrong symptoms, get the wrong solutions;
 
Upvote 0 Downvote
sort() sorts a copy of the original array. Which is why you can do this:

@array = sort @array;

The OP can try using the sort pragma to control the behavior of the sort function and see if it helps, but which one they can use depends on the version of perl they have.

------------------------------------------
- Kevin, perl coder unexceptional! [wiggle]
 
Upvote 0 Downvote
if(open(CSV,$_[0])){
}
else{
print "<HTML><HEAD><TITLE>Error message</TITLE><BODY>\n";
print "<B>Error:</B> Can't open CSV file.\n";
print "<BR>Please, check that you have provided the cgi script with correct CSV file";
print " path in \$CSV_file variable.\n";
print "</BODY></HTML>";
exit(0);
}

@lines=<CSV>;
$lines_length=@lines;
$lines_length=$lines_length-1;
for($l=0;$l<=$lines_length;$l=$l+1){
$lines[$l]=~s/""/%01/g;
while($lines[$l]=~/("[^"]+")/){
$match=$1;
$match=~s/"//g;
$match=~s/\|/%02/g;
$lines[$l]=~s/("[^"]+")/$match/;
}
}
close(CSV);

# @lines = reverse sort (@lines); # works but decending
# @lines = sort (@lines); # hangs then crashes premature

return @lines;
}


#=========================================
 
Upvote 0 Downvote
What's in the CSV file?

Also, as an aside from the sorting issue I'm curious what the purpose of the processing you're carrying out on all the lines is.
 
Upvote 0 Downvote
The file is a flat text file

name|address|category
John|122 Mockingbird|Tools

Header row with about 120 lines under it.

I think the reason the script crashes is because when I'm sorting the heading row is also sorted. But then why doesn't it crash when I reverse sort. I'm telling you this has been driving me nuts for two days. The server I'm on doesn't let me see the error logs really stinks. I was hoping that it was a stupid expression that I was missing or something.

The line processes are part of the total script which allows me to search for a string in the flat file and return the proper record. All of that works fine, except they are displayed in the order they were entered into the file not alphabetically ascending.

By the way, thank you to everyone who helping me with this. I've always been a cut and paste perl guy trying to learn it. Have come a long way, but need a lot more work.



 
Upvote 0 Downvote
having the header in the sort should not be a problem. Print the contents of @lines without doing any sorting and see what it is.

------------------------------------------
- Kevin, perl coder unexceptional! [wiggle]
 
Upvote 0 Downvote
Kevin's right. Please post the actual contents of the @lines array (or a link to it) immediately before it's sorted (i.e. not just an example of what sort of thing could be in it). It's a strange problem you're having and if we can verify that the problem is not in the data, it may be that your Perl installation is faulty and needs to be replaced.
 
Upvote 0 Downvote
I think it must be a perl problem, cause everyone here, and everyone I've talked to says the same thing. If you can reverse sort it, you can sort it.

Is there a way to sort the file itself (database.txt) after I add a new record but keeping the header row in place? I guess I could do it that way since the records wouldn't be changed that often? I know you can open, sort, and close the file but how would one keep the header row there?

 
Upvote 0 Downvote
If the file is initially sorted, using Tie::File would allow you to loop through the file and insert your new record in the appropriate place.

(BTW I've now asked three times to see the data you're trying to sort in order to try to shed some light on your problem. I also wondered what purpose of the processing you're performing on the lines in your CSV file is, so I could possibly suggest an alternative approach. You're not giving me much to work with)
 
Upvote 0 Downvote
Assuming it's a perl problem without posting the data as requested isn't the best assumption. Post the data.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
[noevil]
Travis - Those who say it cannot be done are usually interrupted by someone else doing it; Give the wrong symptoms, get the wrong solutions;
 
Upvote 0 Downvote
to maintain your headers, you could splice the row[0], and then unshift the header row back on to the array, just before closing the file

Paul
------------------------------------
Spend an hour a week on CPAN, helps cure all known programming ailments ;-)
 
Upvote 0 Downvote
If for some security or other reasons you cannot post the data, you can successively remove the last half of the data and try the sort again. If you repeat for about 8 times (120 lines), the program is bound to work. Check what you have last removed to make it work, and you may be surprised to find what's in your data.
 
Upvote 0 Downvote
Are you sure it's failing in the sort, and not somewhere afterwards? Also, seconding what ishnid has already said, there has to be a better way to do what you are trying to do here (like using a proper CSV parser such as Text::CSV_XS for starters).

Steve

[small]"Every program can be reduced by one instruction, and every program has at least one bug. Therefore, any program can be reduced to one instruction which doesn't work." (Object::perlDesignPatterns)[/small]
 
Upvote 0 Downvote
Calling 'reverse sort' calls 'sort' and then reverses the results.

I have to emphasize that (as others alreay have). In order to get your reversed list, the 'sort' is working correctly and passes 'reverse' the sorted list.

If the failure was in the sort algorithm, it would still be present and cause the same problem no matter what you did to the sorted list (reverse in this case).

What happens when you reverse the reversed array ?
Code: 
@lines = reverse reverse sort @lines;

I would direct my attention to the processing of the array by whatever is calling your sub and the assumptions that that code makes about the data in the array.

I notice that this is for a web page, so I'm thinking that there is the possibility that your data contains some funky characters that you are not escaping before inserting which may lead to malformed HTML.

Also, if you're trying to do some search processing, then the script may timeout depending on the size of the array/file, the term you are searching on, and the search algorithm you are using (ie., if you're searching for "zygote", then a linear search will return faster if the array is reverse sorted. Normally not a problem, but in a web environment with a timeout, and the possibility of multiple users accessing the file, you may end up with a whole bunch of race conditions).

 
Upvote 0 Downvote
Status
Not open for further replies.

Similar threads

kokch
  • Locked
  • Question
Need assistance with XML- Sort/Grouping (alpha sort; hide without groups)
Replies
1
Views
242
mikrom
mikrom
tonykent
  • Locked
  • Question
sort by hash key before printing
Replies
3
Views
280
tonykent
tonykent
whn
  • Locked
  • Question
How to sort an all-digit matrix 1
Replies
4
Views
240
rharsh
rharsh
chris01010
  • Locked
  • Question
PERL loop problem
Replies
0
Views
130
chris01010
chris01010
markmulligansr
  • Locked
  • Question
PureData (a.k.a. Netezza, PostgreSQL) ERROR parsing problem
Replies
1
Views
205
PinkeyNBrain
PinkeyNBrain

Part and Inventory Search

Sponsor

Back
Top