Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-11738

Hedged pread takes more time when block moved from initial locations

          Details

          • Type: Bug
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: None
          • Fix Version/s: 2.9.0, 3.0.0-beta1, 2.8.3
          • Component/s: hdfs-client
          • Labels:
            None

            Description

            Scenario :
            Same as HDFS-11708.

            During Hedge read,
            1. First two locations fails to read the data in hedged mode.
            2. chooseData refetches locations and adds a future to read from DN3.
            3. after adding future to DN3, main thread goes for refetching locations in loop and stucks there till all 3 retries to fetch locations exhausted, which consumes ~20 seconds with exponential retry time.

              Attachments

              1. HDFS-11738-branch-2-committed.patch
                13 kB
                Vinayakumar B
              2. HDFS-11738-branch-2.8-committed.patch
                12 kB
                Vinayakumar B
              3. HDFS-11738-05.patch
                12 kB
                Vinayakumar B
              4. HDFS-11738-04.patch
                14 kB
                Vinayakumar B
              5. HDFS-11738-03.patch
                13 kB
                Vinayakumar B
              6. HDFS-11738-02.patch
                14 kB
                Vinayakumar B
              7. HDFS-11738-01.patch
                28 kB
                Vinayakumar B

                Issue Links

                depends upon

                Bug - A problem which impairs or prevents the functions of the product. HDFS-11708 Positional read will fail if replicas moved to different DNs after stream is opened

                • Critical - Crashes, loss of data, severe memory leak.
                • Resolved

                  Activity

                    People

                    • Assignee:
                      vinayrpet Vinayakumar B
                      Reporter:
                      vinayrpet Vinayakumar B
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      8 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: