Tek-Tips is the largest IT community on the Internet today!

Members share and learn making Tek-Tips Forums the best source of peer-reviewed technical information on the Internet!

  • Congratulations Mike Lewis on being selected by the Tek-Tips community for having the most helpful posts in the forums last week. Way to Go!

Round Down to nearest 16th Inch 1

Status
Not open for further replies.

bkrampe

IS-IT--Management
Nov 30, 2006
77
US
I am trying to take a metric number and convert it to a decimal, then round it to the nearest 16th of an inch.

Example 415.131256m -> decimals is 16.34375in.~

Here is were I am running into problems. I cannot seem to find the correct code to round that answer down to 16.3125. Everything I try seems to bring it up to 16.375.

Any help would be greatly appreciated. Thanks in advance.
 
Everything I try…

Please explain, in detail, everything you have tried.

BTW:
1 meter is 39.37008 inches, so your inches is way off!





Skip,
[sub]
[glasses]Just traded in my OLD subtlety...
for a NUance![tongue][/sub]
"The most incomprehensible thing about the universe is that it is comprehensible" A. Einstein

You Matter...
unless you multiply yourself by the speed of light squared, then...
You Energy!
 
Looks to me [tt]415.131256[/tt] m = [tt]16343.75023622[/tt] in [ponder]
16.34375 in is actually less than 0.5m

Or am I missing something here...

---- Andy

"Hmm...they have the internet on computers now"--Homer Simpson
 
Sorry I meant to say 415.131256 millimeters mm -> decimals is 16.34375in.~

currently I have...

cast((round((itmdim.dimx/25.4)*16,0)/16) as decimal(10,4)) as [dimX (in)]
 
FLOOR(itmdim.dimx*16)/16


Did this in Excel using formula =INT(A1*16)/16, interested in the VALUE only and not the format...
Value. Round down
16 16
16.048 16
16.096144 16.0625
16.14443243 16.125
16.19286573 16.1875
16.24144433 16.1875
16.29016866 16.25
16.33903917 16.3125
16.38805628 16.375
16.43722045 16.375
16.48653211 16.4375
16.53599171 16.5
16.58559968 16.5625
16.63535648 16.625
16.68526255 16.625
16.73531834 16.6875
16.7855243 16.75
16.83588087 16.8125
16.88638851 16.875
16.93704768 16.875
16.98785882 16.9375

Skip,
[sub]
[glasses]Just traded in my OLD subtlety...
for a NUance![tongue][/sub]
"The most incomprehensible thing about the universe is that it is comprehensible" A. Einstein

You Matter...
unless you multiply yourself by the speed of light squared, then...
You Energy!
 
I would try it this way:

Given: 415.131256 mm

One inch is 25.4 mm, so one sixteenth of the inch is 25.4/16 = 1.5875 mm

Then, how many sixteenth of the inch is 415.131256 mm ?
It is:
floor(415.131256 / (25.4/16)) = 261

Now, when 415.131256 mm = 261 sixteenth of the inch, then in inches it is:
261 / 16 inch = 16.3125 inch
 
I tried this (in DB2, because i don't have Microsoft SQL):
Code:
select
  415.131256 as mm,
  cast(
    cast(floor(415.131256 / cast(25.4/16 as decimal(5,4))) as decimal(10,4)) / 16.0 
  as decimal(10, 4)) as inch
from sysibm.sysdummy1
;

....+....1....+....2....+.     
        MM           INCH      
415.131256        16.3125      
********  End of data  ********
 
This was the syntax that I ended up using to get my answer. I also discovered that in order for me to get consistent answers, I had to round my Metric # to 4 decimal places.

floor([dimX]/1.5875) * 0.0625 as FinalValue

Thanks everyone for their help!
 
Status
Not open for further replies.

Part and Inventory Search

Sponsor

Back
Top