Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6227

ShortCircuitCache#unref should purge ShortCircuitReplicas whose streams have been closed by java interrupts

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.0
    • Fix Version/s: 2.5.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      While running tests in a single node cluster, where short circuit read is enabled and multiple threads may read the same file concurrently, one of the read got ClosedChannelException and failed. Full exception trace see comment.

      1. ShortCircuitReadInterruption.test.patch
        5 kB
        Jing Zhao
      2. HDFS-6227.002.patch
        7 kB
        Colin Patrick McCabe
      3. HDFS-6227.001.patch
        7 kB
        Colin Patrick McCabe
      4. HDFS-6227.000.patch
        3 kB
        Jing Zhao

        Activity

        Jing Zhao created issue -
        Jing Zhao made changes -
        Field Original Value New Value
        Summary ShortCuit Read failed due to ClosedChannelException Short circuit read failed due to ClosedChannelException
        Jing Zhao made changes -
        Attachment ShortCircuitReadInterruption.test.patch [ 12646134 ]
        Colin Patrick McCabe made changes -
        Assignee Colin Patrick McCabe [ cmccabe ]
        Jing Zhao made changes -
        Attachment HDFS-6227.000.patch [ 12646406 ]
        Colin Patrick McCabe made changes -
        Attachment HDFS-6227.001.patch [ 12646418 ]
        Colin Patrick McCabe made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Colin Patrick McCabe made changes -
        Summary Short circuit read failed due to ClosedChannelException ShortCircuitCache#unref should purge ShortCircuitReplicas whose streams have been closed by java interrupts
        Colin Patrick McCabe made changes -
        Attachment HDFS-6227.002.patch [ 12646643 ]
        Jing Zhao made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Fix Version/s 2.5.0 [ 12326264 ]
        Resolution Fixed [ 1 ]
        Karthik Kambatla (Inactive) made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Colin Patrick McCabe
            Reporter:
            Jing Zhao
          • Votes:
            0 Vote for this issue
            Watchers:
            12 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development