Floating point division round-off error

XMLWordPrintable

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Low
    • Code Generation Tools
    • CODEGEN-10343
    • Hide
      C2000_16.9.0.LTS
      C2000_18.1.0.LTS
      C2000_22.6.0.LTS
      C2000_25.11.0.LTS
      C2000_15.12.0.LTS
      C2000_6.4.0
      C2000_21.6.0.LTS
      C2000_18.12.0.LTS
      C2000_20.2.0.LTS
      Show
      C2000_16.9.0.LTS C2000_18.1.0.LTS C2000_22.6.0.LTS C2000_25.11.0.LTS C2000_15.12.0.LTS C2000_6.4.0 C2000_21.6.0.LTS C2000_18.12.0.LTS C2000_20.2.0.LTS
    • default

      The attached source file has these statements ...

          float A = 1.002002954483032F; // as U32 -> 0x3F8041A2
          float B = 0.999000012874603F; // as U32 -> 0x3F7FBE77
          float C = A/B;
      

      The result computed into C is hex: 0x3f80627f float: 1.003005862236023 . When the same code is run on a typical x86 machine/compiler, the result is hex: 0x3f806280 float: 1.003005981445313.

      The same result is seen whether the DIVF32 instruction, or the RTS function FS$$DIV is used.

      Here is a typical build command ...

      "C:/ti/compilers/ti-cgt-c2000_22.6.0.LTS/bin/cl2000" -v28 -ml -mt --cla_support=cla2 --float_support=fpu32 --tmu_support=tmu0 --vcu_support=vcu0 --fp_mode=strict --include_path="C:/ti/compilers/ti-cgt-c2000_22.6.0.LTS/include" --include_path="C:/ti/compilers/ti-cgt-c2000_22.6.0.LTS/lib/src" -g --diag_warning=225 --diag_wrap=off --display_error_number --abi=coffabi --src_interlist --preproc_with_compile --preproc_dependency="main.d_raw"  "../main.c"
      

            Assignee:
            TI User
            Reporter:
            TI User
            Votes:
            7 Vote for this issue
            Watchers:
            5 Start watching this issue

              Created:
              Updated:

                Connection: Intermediate to External PROD System
                EXTSYNC-3477 - Floating point division round-off e...
                SYNCHRONIZED
                • Last Sync Date: