PRU RPMsg swaps which message is sent to which core

XMLWordPrintable

    • Type: Bug
    • Resolution: Unresolved
    • Priority: Medium
    • Linux Core SDK
    • LCPD-42167
    • 08.06.00
    • 12.00
    • Hide
      am335x-evm
      am437x-sk
      am571x-idk
      am572x-idk
      am62xx_sk-fs
      am64xx-evm
      am64xx_sk-fs
      am654x-idk
      beaglebone-black
      Show
      am335x-evm am437x-sk am571x-idk am572x-idk am62xx_sk-fs am64xx-evm am64xx_sk-fs am654x-idk beaglebone-black

      When two separate applications are sending RPMsg messages to two separate PRU cores, sometimes the messages get swapped (i.e., sometimes the message for PRU0 goes to PRU1, and sometimes the message for PRU1 goes to PRU0). From the customer report, it is unclear if the messages are truly swapped (i.e., PRU0 gets PRU1's message and PRU1 gets PRU0's message AT THE SAME TIME), or if one core's message simply gets overwritten (i.e., PRU0's message goes to both PRU0 and PRU1).

      Customer confirmed that if they removed the "static" from this line, then the behavior was no longer observed:
      drivers/rpmsg/rpmsg_pru.c, function rpmsg_pru_write()
      static char rpmsg_pru_buf[FIFO_MSG_SIZE];

      rpmsg_pru.c used in ti-linux-kernel-6.6.y is virtually unchanged for the past 8 years, so it is very different from the upstreamed rpmsg drivers. It might make the most sense for this bug to be fixed as a part of a driver rewrite to enable upstreaming. The issue may be fixed in ti-linux-kernel-6.12.y since we moved PRU cores to use the generic rpmsg_char.c starting in this kernel release (see https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1494495/faq-am62x-am64x-how-to-enable-pru-rpmsg-on-processor-sdk-linux-11-0). But we still need to test and verify.

      test information:
      https://e2e.ti.com/support/processors-group/processors/f/processors-forum/1430451/am6412-rpmsg-echo-example-of-the-pssp-occurs-occasional-interference-when-communicating-with-two-prus-simultaneously
      Run 2 different shell commands in 2 different ssh processes:

      1. pru0
        while true; do echo -n "$(date -Ins): "; echo "test30" > /dev/rpmsg_pru30; timeout 0.05 cat /dev/rpmsg_pru30; done
      1. pru1
        while true; do echo -n "$(date -Ins): "; echo "test31" > /dev/rpmsg_pru31; timeout 0.053 cat /dev/rpmsg_pru31; done

      Customer's test results
      Before static removal:
      PRU0 and PRU1 have 3 swaps in a total of 828 RPMsg communications
      (Occurs 3 times in 26 seconds)

      After static removal: Occurrence
      pru0 and pru1 have 0 swaps in a total of 227279 RPMsg communications
      (0 occurrences in 122 minutes)

            Assignee:
            TI User
            Reporter:
            TI User
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

              Created:
              Updated:

                Connection: Intermediate to External PROD System
                EXTSYNC-5849 - PRU RPMsg swaps which message is se...
                SYNCHRONIZED
                • Last Sync Date: