Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

precision in C

Status
Not open for further replies.

EvLer

Programmer
Aug 27, 2008
5
US
Hello everyone,

I need to be able to limit(or control) precision of double datatype to 10 decimal places and not more or less. How would i do it? I can also use float or any other datatype, the issue is making precision only 10 decimal places:

0.0123456789

and this is not output formatting.... I am working with a program that does a scientific computation, so precision is important in calculations, not for the display.

Thank you in advance.
 
Guess you need to work in a language like Coral66, Ada or Cobol where the number of decimal places can be fixed.

Assuming fmod behaves the same way as % and that they haven't done it properly for -ve numbers, after each calculation, you could do something like
Code:
x -= fmod (x, 1E-10);
Very expensive though.
 
Neat!

EvLer,

If the number is greater than or equal to 1 ( or less than 0.1) how should it be rounded? For example, does

1.12345123451 get rounded to
1.1234512345 (10 digits after decimal) or
1.123451234 (10 significant figures) ?
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top