Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14870

[SBN read] Client Settings to ObserverReadProxyProvider read a file request, due to the file atime time needs to be modified, lead to request to be processed twice

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 2.10.0, 3.3.0, 3.2.1
    • Fix Version/s: None
    • Component/s: hdfs
    • Labels:
      None

      Description

      Client Settings agent refers to ObserverReadProxyProvider class, read the RPC request will be forwarded to the ONN, but involves the need to modify the file, the request will be processed twice, cause the request time is too long;

      When ONN received from the client to read a file, will call FSNamesystem getBlockLocations method to get the list of file information, and the current atime decided the current file needs to be updated (dfs.namenode.accesstime.precision, HDFS file access time is accurate, The default value is 1 hour). The ONN does not currently support write operations. This logic throws StandbyException and the client retries the RPC request and forwards it to ANN for execution. As a result, the current file getBlockLocations method will be called twice, lengthening the entire RPC request.

       

       

        Attachments

        1. test_case
          6 kB
          huhaiyang

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              haiyang Hu huhaiyang
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated: