![]() Previous |
![]() Next |
You cannot expect exact equality between SHORTDECIMAL
and DECIMAL
or NUMBER
representations of a decimal number with a fractional component, because the DECIMAL
and NUMBER
data types have more significant digits to approximate fractional components that cannot be represented exactly.
Suppose you define a variable with a SHORTDECIMAL
data type and set it to a fractional decimal number, then compare the SHORTDECIMAL
number to the fractional decimal number, as shown here.
DEFINE sdvar SHORTDECIMAL sdvar = 1.3 SHOW sdvar EQ 1.3
The comparison is likely to return NO
. What happens in this situation is that the literal is automatically typed as DECIMAL
and converts the SHORTDECIMAL
variable sdvar
to DECIMAL
, which extends the decimal places with zeros. A bit-by-bit comparison is then performed, which fails. The same comparison using a variable with a DECIMAL
or a NUMBER
data type is likely to return YES
.
There are several ways to avoid this type of comparison failure:
Do not mix the SHORTDECIMAL
with DECIMAL
or NUMBER
types in comparisons. To avoid mixing these two data types, generally avoid defining variables with decimal components as SHORTDECIMAL
.
Use the ABS
or ROUND
function to allow for approximate equality. The following statements both produce YES
.
SHOW ABS(sdvar - 1.3) LT .00001 SHOW ROUND(sdvar, .00001) EQ ROUND(.3, .00001)