Click or drag to resize

CalculationPrecisionStrategy Enumeration

Enumerates strategies for handling calculation precision. Because of the precision issue of IEEE 754 Floating-Point Arithmetic, some "seemingly simple" formulas may not be calculated as the expected result. Such as formula "=-0.45+0.43+0.02", when calculating operands by '+' operator directly, the result is not zero. For such kind of precision issue, some special strategies may give the expected result.

Namespace:  Aspose.Cells
Assembly:  Aspose.Cells (in Aspose.Cells.dll) Version: (20.10)
public enum CalculationPrecisionStrategy
  Member nameValueDescription
None0 No strategy applied on calculation. When calculating just use the original double value as operand and return the result directly. Most efficient for performance and applicable for most cases.
Round1 Rounds the calculation result according with significant digits.
Decimal2 Uses decimal as operands when possible. Most inefficient for performance.
See Also