Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9887

WebHdfs socket timeouts should be configurable

    Details

    • Target Version/s:
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Added new configuration options: dfs.webhdfs.socket.connect-timeout and dfs.webhdfs.socket.read-timeout both defaulting to 60s.
    • Flags:
      Patch

      Description

      WebHdfs client connections use sockets with fixed timeouts of 60 seconds to connect, and 60 seconds for reads.
      This is a problem because I am trying to use WebHdfs to access an archive storage system which can take minutes to hours to return the requested data over WebHdfs.

      The fix is to add new configuration file options to allow these 60s defaults to be customised in hdfs-site.xml.

      If the new configuration options are not present, the behavior is unchanged from before.

        Attachments

        1. HADOOP-12827.004.patch
          11 kB
          Chris Douglas
        2. HADOOP-12827.003.patch
          11 kB
          Chris Douglas
        3. HADOOP-12827.002.patch
          11 kB
          Austin Donnelly
        4. HADOOP-12827.002.patch
          11 kB
          Austin Donnelly
        5. HADOOP-12827.002.patch
          11 kB
          Austin Donnelly
        6. HADOOP-12827.001.patch
          6 kB
          Austin Donnelly

          Issue Links

            Activity

              People

              • Assignee:
                and1000 Austin Donnelly
                Reporter:
                and1000 Austin Donnelly
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: