Opened 5 years ago

Closed 4 years ago

#257 closed defect (fixed)

numeric_to_ascii_base_ can return numbers including digits not in that base

Reported by: Gary Dixon Owned by: Eric Swenson
Priority: minor Milestone: MR12.8
Component: General Version: MR12.8
Keywords: Cc:

Description

A test program I wrote calls a new cv_fixed_point_integer_$decimal subroutine to convert an input string to a float dec(59) number; and then converts this number back to a numeric string in the same base. This two-way conversion measure precision loss that can occur in some odd numeric bases. However, the loss of precision is not the issue of this ticket.

For certain float dec(59) input numbers and output base values, numeric_to_ascii_base_ may emit a numeric string containing digits which are not valid for the specified output base. Examples follow for desired output bases 14 and 15.

tcfps$test_decimal    in: "0.01"
  details: T         out:  5.1020408163265306122448979591836734693877551020408163265306e-3
     base: 14        out:  0.00e

tcfps$test_decimal    in: "0.01"
  details: T         out:  4.4444444444444444444444444444444444444444444444444444444444e-3
     base: 15        out:  0.00f

Notice that base 14 output ends with an "e" digit where the highest valid digit in base 14 is "d".
Base 15 output ends with an "f" digit where the highest valid digit in base 15 is "e".

In each example above, the first out: value gives the float dec(59) input to numeric_to_ascii_base_ as expressed in decimal by the numeric_to_ascii_ subroutine.

I'm guessing this may be an "off-by-1" bug. For an input value just shy of a 1 in the last digit position, numeric_to_ascii_base_ adds another digit position to the right of the radix point, with a value in that position of the highest digit valid for that numeric base... but it selects a digit 1-higher than the highest valid digit by mistake.

Change History (3)

comment:1 by Gary Dixon, 4 years ago

This problem was corrected by changes to numeric_to_ascii_base_.pl1 program:

  1) change(2022-03-17,GDixon), approve(2022-07-13,MCR10101), audit(2022-07-27,Swenson):
      A) Revise numeric_to_ascii_base_ to use more accurate calculation
         algorithm, and to have a longer return value.  The original
         subroutine returned a char(72) varying string, which is not long
         enough to hold all digits of binary and quaternary numbers.
      B) Replace code in numeric_to_ascii_ subroutine (originally a separate
         source module) with code here so it can share the output formatting
         routine developed for numeric_to_ascii_base_.
  2) change(2022-04-08,GDixon), approve(2022-07-13,MCR10101), audit(2022-07-27,Swenson):
      A) Use decimal_no_round_$divide to eliminate rounding errors when
         calculating non-decimal base strings.
  3) change(2022-04-18,GDixon), approve(2022-07-13,MCR10101), audit(2022-07-27,Swenson):
      A) Change do_fraction algorithm to round-up the product to offset
         inaccuracies due to limited-precision arithmetic.

comment:3 by Eric Swenson, 4 years ago

Resolution: fixed
Status: newclosed
Version: MR12.8

Fixed with MR12.8-1030.

Note: See TracTickets for help on using tickets.