Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6214

Webhdfs has poor throughput for files >2GB

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.0.0-alpha, 3.0.0-alpha1
    • 2.5.0
    • webhdfs
    • None
    • Reviewed

    Description

      For the DN's open call, jetty returns a Content-Length header for files <2GB, and uses chunking for files >2GB. A "bug" in jetty's buffer handling results in a ~8X reduction in throughput.

      Attachments

        1. HDFS-6214.patch
          3 kB
          Daryn Sharp

        Activity

          People

            daryn Daryn Sharp
            daryn Daryn Sharp
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: