Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-7163

WebHdfsFileSystem should retry reads according to the configured retry policy.

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.5.1, 3.0.0-alpha1
    • 2.8.0, 2.7.3, 3.0.0-alpha1
    • webhdfs
    • None

    Description

      In the current implementation of WebHdfsFileSystem, opens are retried according to the configured retry policy, but not reads. Therefore, if a connection goes down while data is being read, the read will fail and the read will have to be retried by the client code.

      Also, after a connection has been established, the next read (or seek/read) will fail and the read will have to be restarted by the client code.

      Attachments

        1. HDFS-7163.001.patch
          23 kB
          Eric Payne
        2. HDFS-7163.002.patch
          24 kB
          Eric Payne
        3. HDFS-7163.003.patch
          24 kB
          Eric Payne
        4. HDFS-7163.004.patch
          25 kB
          Eric Payne
        5. HDFS-7163.005.patch
          29 kB
          Eric Payne
        6. HDFS-7163-branch-2.003.patch
          23 kB
          Eric Payne
        7. HDFS-7163-branch-2.004.patch
          24 kB
          Eric Payne
        8. HDFS-7163-branch-2.7.003.patch
          23 kB
          Eric Payne
        9. HDFS-7163-branch-2.7.004.patch
          24 kB
          Eric Payne
        10. HDFS-7163-branch-2.7.005.patch
          28 kB
          Eric Payne
        11. WebHDFS Read Retry.pdf
          46 kB
          Eric Payne

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            epayne Eric Payne
            epayne Eric Payne
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment