Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations John Tel on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Precision of the floating-point types?

Status
Not open for further replies.

Bobby2004

Programmer
Mar 10, 2004
82
IN
I need to find out the precision of the floating-point data types viz. Single, Double, and Decimal i.e. the maximum number of decimal digits ( i.e. base 10 digits ) that may be specified in thier literals. As this is not documented I thought I'd keep on enterring ever larger numbers in the Immediate Window until I got an overflow error and then count the number of digits. However to my frustration, as soon as I exceed 15 9' i.e. 9999999999999999, it gets converted to 1E+16.

How can I find out the precision and scale of these three data types?
 
Um....I notice that those are .Net datatypes, not VB6 datatypes. Here are the VB6 ones:
The .Net has several differences, the chief of which are:
1. Object replaces (roughly) Variant.
2. Integer is 4 bytes instead of 2. The 2 byte integer is called Short.
3. Long is 8 bytes instead of 4.
4. Currency (scaled integer, meaning arbitrary decimal point set at 4 and only 4 places, therefore NOT floating point) is gone.

HTH

Bob
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top