Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations SkipVought on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

VB Editor truncates number - Why

Status
Not open for further replies.

porto99

Technical User
Nov 1, 2004
96
0
0
GB
'Consider the following code:-

Dim Value_1 As Integer
Dim Value_2 As Single
Dim Value_3 As Double
Dim Value_4 As Long

' If I type 1.012345678901234567890 it always truncates!

Value_1 = 1.01234567891237

Value_2 = 1.01234567891236

Value_3 = 1.01234567891236

Value_4 = 1.01234567891235
 
i think it rounds it (not truncates) because that's the largest precision stadard VB datatypes may handle.
 
Actually, they convert to
Code:
Value_1 = 1
Value_2 = 1.012346
Value_3 = 1.01234567891236
Value_4 = 1
The "whole number" data types (Integer & Long) are rounded to the nearest integer values. "Single" has a precision of 7 digits and double has a 15 digit precision. If you need to represent "1.012345678901234567890" then try
Code:
Dim Value_6 As Variant
Value_6 = cDec(1.012345678901234567890)
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top