C and DBMS 1

[andrealds]

Hi Everyone,

I´m developing a project where I have my information in TXT files (It´s about 400 files with 6000 lines each).

With this amount of Information, i need to make some math calculations (Averages, Regressios, Statistic Calcs etc).

My first shot was to make it in VBA for Excel but it´s really slow. So i´m looking for another language to make the analisys. Im thinking about making it in C as I already have some experience in it.

My question is: Is it interesting just to analize the data direct from the TXT files and generate TXT files for the conclusions I need, or is it interesting to use some DBMS like MySql, PostgreSql etc. If yes, which one would you suggest (preferred one embedded with C) ?

Thanks,
André.

 

[Salem]

> or is it interesting to use some DBMS like MySql, PostgreSql etc.
Interesting to who, you or us?

Personally, I would do this in Perl.

But if you want to use this as an exercise to leverage some knowledge of say mySql, then go for it.


--
If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.

 

[andrealds]

Thank you for your help Salem and I´m Sorry if i wasnt clear in some parts.

What I meant by
"or is it interesting to use some DBMS like MySql, PostgreSql etc."
Was:
Would it be better for any reason (Speed, Easy in manipulating data, organization or whatever) if I use a DBMS together with the language I choose ?? Or Would it be better if I work direct with the TXT files ?

If I should use a DBMS together, which one should I use that would help ME better?

Thank you for your cooperation,
André.

 

[Salem]

> Would it be better for any reason (Speed, Easy in manipulating data, organization or whatever)
> if I use a DBMS together with the language I choose ??
> Or Would it be better if I work direct with the TXT files ?
I'm sure an argument could be justified for either case.
But then again, you've got your entire work experience to date, and your own career ambitions on which to base a decision.
All we've got is a couple of hundred words at most.

> If I should use a DBMS together, which one should I use that would help ME better?
And how exactly are you expecting us to know what "ME" wants out of this exercise?

Sure, being able to parse files in C, perform calculations and write the results into a DBMS might be worth knowing for it's own sake. But is it something you can make a career out of - I've no idea.

It seems to me you've already made the decision, and just seeking someone else to validate it in some way.

If you're asking if your "C/DBMS" approach is doomed to fail, then I would say that "It isn't".
But I would also say that it'll take a lot longer to write and debug than you might expect, if you've never done that kind of thing before.

Like I said, I'd to it in perl and write the results out as a CSV file. Any spreadsheet or DBMS would have no trouble importing that data, if that's what I thought the next step would be.

But if you're developing a product for people with no technical skill, who want an end-to-end solution, then maybe you have to go the extra mile and write the results into the database.


--
If you dance barefoot on the broken glass of undefined behaviour, you've got to expect the occasional cut.