Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations IamaSherpa on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Problem with float precision... help!!

Status
Not open for further replies.

artcalv

Programmer
Apr 21, 2007
1
BE
I'm running GAWK on WIN32 and I have a terrible problem with float precision.

I read from a file float numbers with 9 decimals, and I've checked it is ok. The problem is when I try to operate with them, because they become just 6 decimals precision.

What can I do?? I've tried with printf("%.8f",res) but it prints all the time "0.80.80.80.80.8...".

Here is my code. Thanks a lot!!

gawk "{post=$1}{print post-prev}{prev=$1}" myInput
 
Hi

To avoid misunderstandings, you use the [tt]gawk[/tt] from CygWin's terminal, so from [tt]bash[/tt] running in a Command Prompt window. It seems to work for me.
Code:
[blue]master #[/blue] cat pi.txt
3.14159265358979323846264338327950288419716939937510

[blue]master #[/blue] awk '{printf("%.8f\n",$1)}' pi.txt
3.14159265

[blue]master #[/blue] awk '{printf("%.8f\n",$1*2)}' pi.txt
6.28318531
Show us more from your code and how actually run it.

Feherke.
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top