Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-894

dfs client protocol should allow asking for parts of the block map

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: 0.14.0
    • Component/s: None
    • Labels:
      None

      Description

      I think that the HDFS client protocol should change like:

      /** The meta-data about a file that was opened. */
      class OpenFileInfo {
      /** the info for the first block */
      public LocatedBlockInfo getBlockInfo();
      public long getBlockSize();
      public long getLength();
      }

      interface ClientProtocol extends VersionedProtocol {
      public OpenFileInfo open(String name) throws IOException;
      /** get block info for any range of blocks */
      public LocatedBlockInfo[] getBlockInfo(String name, int blockOffset, int blockLength) throws IOException;
      }

      so that the client can decide how much block info to request and when. Currently, when the file is opened or an error occurs, the entire block list is requested and sent.

        Attachments

        1. partialBlockList2.patch
          50 kB
          Konstantin Shvachko
        2. partialBlockList3.patch
          72 kB
          Konstantin Shvachko
        3. partialBlockList6.patch
          72 kB
          Konstantin Shvachko

          Activity

            People

            • Assignee:
              shv Konstantin Shvachko
              Reporter:
              omalley Owen O'Malley
            • Votes:
              1 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: