Uploaded image for project: 'Embedded Software & Tools'
  1. Embedded Software & Tools
  2. EXT_EP-10879

Floating point division round-off error

XMLWordPrintable

    • Icon: Bug Bug
    • Resolution: Unresolved
    • Icon: Low Low

      The attached source file has these statements ...

          float A = 1.002002954483032F; // as U32 -> 0x3F8041A2
          float B = 0.999000012874603F; // as U32 -> 0x3F7FBE77
          float C = A/B;
      

      The result computed into C is hex: 0x3f80627f float: 1.003005862236023 . When the same code is run on a typical x86 machine/compiler, the result is hex: 0x3f806280 float: 1.003005981445313.

      The same result is seen whether the DIVF32 instruction, or the RTS function FS$$DIV is used.

      Here is a typical build command ...

      "C:/ti/compilers/ti-cgt-c2000_22.6.0.LTS/bin/cl2000" -v28 -ml -mt --cla_support=cla2 --float_support=fpu32 --tmu_support=tmu0 --vcu_support=vcu0 --fp_mode=strict --include_path="C:/ti/compilers/ti-cgt-c2000_22.6.0.LTS/include" --include_path="C:/ti/compilers/ti-cgt-c2000_22.6.0.LTS/lib/src" -g --diag_warning=225 --diag_wrap=off --display_error_number --abi=coffabi --src_interlist --preproc_with_compile --preproc_dependency="main.d_raw"  "../main.c"
      

            syncuser TI User
            syncuser TI User
            Votes:
            6 Vote for this issue
            Watchers:
            4 Start watching this issue

              Created:
              Updated: