floor w/ and w/o units gives different results

[Jwouters]

Hi all, I'm pretty new to Mathcad (I'm still young ), but I have run across a pretty weird problem. I did try google/help/eng-tips without any luck.

I think my problem is best explained in the attached file but here it is:

My 'goal' is to floor a division incluing units m/mm. In my example, I divide 12.75m by 1275mm which should result in 10. But Mathcad actually return 9.99....998. 'flooring' this number would result in 9, where 10 is expected.

When I try to floor(12750/1275) I get 10, as expected.

I would like to know if there is any solution to this problem or if I'm just stuck with this error.

 

[jhardy1]

@Jwouters:
The issue with using ANY software is to understand its constraints and limitations, and be practical in how you use it; in particular, be aware that digital computers have a finite precision, and that there is a difference between "exactly equal to" and "close enough for all practical purposes".
For example, IRStuff suggested using a tolerance of 10^-10 m (one Angstrom); if you are building a bridge, or planning a trip across the country, that should be "near enough". But what if your calculation relates to the "target" of the Large Hadron Collider? Now you have to hit a target which is the size of a nucleus - MUCH smaller than one Angstrom - so you would have to have a MUCH finer tolerance setting! Check out the definition of the "barn" as a unit of area in nuclear physics:

http://en.wikipedia.org/wiki/Barn_(unit)

- and yes, Mathcad does know how big (small!) 1 barn is!)

http://julianh72.blogspot.com

 

[Jwouters]

@jhardy1:
I'm now indeed more aware of the constraints/limitations of software like this. I just found it out the hard way

Funny thing is, I was thinking about the LHC as well. Imagine 9,99...98 vs 10 being the difference of 'finding a Higgs-Boson particle' vs 'finding some random particle' ... I guess (hope) those scientist are well aware of these 'errors'

 

[IRstuff]

It's incumbent on the engineer, or scientist, to understand the nature of his problem, his solution, and his tool. While Mathcad does its normal numeric processing with double precision, quad precision math has been around and available for decades, and Mathcad's symbolic processor has arbitrary precision.

With Mathcad's symbolic processor, you could crank barns, outhouses, and sheds to the 10th power and beyond all day long.

TTFN
faq731-376

 

[jn196]

Because of this type of rounding issues, well known in programming, when trimming a number, say X, we use X + epsilon instead.

Mathcad's TOL = 0.001 works as epsilon, and to simplify consider the alternative function
jfloor(X) = floor(abs(X) + TOL)* sign(X)

We keep these sort of things in a standardized include file.