limit csv to 199 fields 1

[travs69]

I'm parsing through a csv file one line at a time and I need to check each line and truncate anything over 199 entries. I would prefer to stick with the core libraries because if possible!

@tmp = split /\,/, $_;
$count = $#tmp <= 198 ? $#tmp : 198;
$line = "";
for $num (0..$count) {
$line .= "$tmp[$num],";
}

print "$line\n";





~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Travis - Those Who Say It Cannot Be Done Are Usually Interrupted by Someone Else Doing It

 

[MillerH]

[url=http://perldoc.perl.org/functions/my.html][black][b]my[/b][/black][/url] [blue]@tmp[/blue] = [url=http://perldoc.perl.org/functions/split.html][black][b]split[/b][/black][/url] [red]/[/red][purple][purple][b]\,[/b][/purple][/purple][red]/[/red], [blue]$_[/blue][red];[/red]

[blue]$#tmp[/blue] = [fuchsia]198[/fuchsia] [olive][b]if[/b][/olive] [blue]$#tmp[/blue] > [fuchsia]198[/fuchsia][red];[/red] [gray][i]# Truncate[/i][/gray]

[black][b]my[/b][/black] [blue]$line[/blue] = [url=http://perldoc.perl.org/functions/join.html][black][b]join[/b][/black][/url] [red]'[/red][purple],[/purple][red]'[/red], [blue]@tmp[/blue][red];[/red]

[url=http://perldoc.perl.org/functions/print.html][black][b]print[/b][/black][/url] [red]"[/red][purple][blue]$line[/blue][purple][b]\n[/b][/purple][/purple][red]"[/red][red];[/red]

- Miller

 

[travs69]

Thanks Miller!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Travis - Those Who Say It Cannot Be Done Are Usually Interrupted by Someone Else Doing It

 

[MillerH]

Your welcome.

Also, since you like regex's so much, I might as well share a way to do the same in a single line. I would advise against using this method as it is a lot more obfuscated, but you might find it amusing:

[red]s/[/red][purple]((?:,.*?){198}),.*[/purple][red]/[/red][purple][blue]$1[/blue][/purple][red]/[/red][red];[/red]
[url=http://perldoc.perl.org/functions/print.html][black][b]print[/b][/black][/url] [red]"[/red][purple][blue]$_[/blue][purple][b]\n[/b][/purple][/purple][red]"[/red][red];[/red]

- Miller

 

[KevinADC]

your original code was very good travs69:

@tmp = split /\,/, $_;
$count = $#tmp <= 198 ? $#tmp : 198;
$line = "";
for $num (0..$count) {
 $line .= "$tmp[$num],";
}

print "$line\n";

Sure it can be written a little more succinctly, but your code was solid. What was your question really? Just hoping to see if your code was OK?

------------------------------------------
- Kevin, perl coder unexceptional!

 

[stevexff]

This works fine if you know that the input doesn't have any quoted fields with embedded commas. For a more generic solution, I'd advise Text::CSV, although this doesn't meet your requirement to use base libraries only. About the worst case scenario is a row like

field1,"field 2a, field 2b","field3a,field3b ""with embedded quotes""",field4

The other problem is that there is no real 'standard' format for a CSV. Most programs quote strings with commas, and double the quotes inside other quotes.

Steve

[small]"Every program can be reduced by one instruction, and every program has at least one bug. Therefore, any program can be reduced to one instruction which doesn't work." (Object:erlDesignPatterns)[/small]

 

[ishnid]

Steve makes a good point. Plus the Text::CSV module is pure-Perl so it's easy to install, even if you have restricted privileges wherever you're running this code.

 

[travs69]

It isn't a matter of being easy to install, if I want something it's somebody else's issue to make it happen even if it means re-compiling 30 other modules to make it happen (which is nice really), it's just all the paper work I have to go through and all the re-testing


Kevin - Basically yeah.. I wasn't sure if there was an easier way that I was missing. 99.9% of the time I can make it work no matter what.. but I like to learn things along the way so I post stuff here just to see how others would do it.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Travis - Those Who Say It Cannot Be Done Are Usually Interrupted by Someone Else Doing It

 

[ishnid]

Fair enough, though with pure Perl modules there's no compiling at all. You'd just have to be able to upload files to wherever you're running your scripts. You know better than me what kind of permissions you have, of course.

 

[travs69]

It's not a permissions issue as much as it is a political/safety issue. I can't do anything that would constitute installing new software. If I want a new module I have to fill out the paper work. One of the guys in another dept installs it on a test system. Make sure it works okay, if it doesn't he does what has to be done to make it work (dependencies, or re-compiling other modules if needed). Then I have to schedule a time for the production system when there is no other work going on that could possible affect anyone else (usually 4-8 weeks out). Then he will install it in production. Then I can use it. Sounds like fun huh!

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Travis - Those Who Say It Cannot Be Done Are Usually Interrupted by Someone Else Doing It

 

[MillerH]

Btw, Kevin was right in that your original code would have worked fine. You would simply have had to chop off the last comma since you weren't using join.

I understand the problems of bureaucracy. They essentially discourage what they deem "unsafe" types of change, and therefore make more work in the long run. It's too bad that Text::CSV and Text::CSV_XS are not yet part of the core libraries for perl, as they are essentially a fundamental need for a lot of different projects.

Even if you don't use Text::CSV_XS or Text::CSV in this project, I would advise you to start the "paperwork" now. I expect that there are probably other modules that should be considered core libraries but are not part of the default installation of perl. It's about building up a list of tools to have at your disposal later on.

Anyway, that's enough of my soapboxing

- Miller

PS,
Simply by the tech guys a case of beer. That'll make them a lot more agreeable.

Oh wait, their geeks. Get them some pocket protectors.

 

[stevexff]

They essentially discourage what they deem "unsafe" types of change

Indeed. They are never happy to install a widely used, tested, open source module from a globally recognised source like CPAN. But it is always OK to install something you knocked out in an afternoon, based on a design you wrote on the back of a fag packet...

Steve

[small]"Every program can be reduced by one instruction, and every program has at least one bug. Therefore, any program can be reduced to one instruction which doesn't work." (Object:erlDesignPatterns)[/small]

 

[MillerH]

Most bureaucracy aren't actually paying to get things done, they're paying to hold someone accountable. Therefore all these hoops that you have to jump through are just discouraging you from passing the accountability on to someone else, even if it is a comparatively minor amount of risk.

It takes a competent CTO to make sure the trend doesn't lean too far in the avoid-risk mindset. And unfortunately, it sounds like travs is in an environment that doesn't effectively measure risk.

- Miller