Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9110

Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for better efficiency

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.7.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:

      Description

      This is a request to do some cosmetic improvements on top of HDFS-8480. There a couple of File -> java.nio.file.Path conversions which is a little bit distracting.

      The second aspect is more around efficiency, to be perfectly honest I'm not sure what the number of files that may be processed. However as HDFS-8480 eludes to it appears that this number could be significantly large.

      The current implementation is basically a collect and process where all files first is being examined; put into a collection and after that processed. HDFS-8480 could simply be further enhanced by employing a single iteration without creating an intermediary collection of filenames by using a FileWalker

        Attachments

        1. HDFS-9110.00.patch
          2 kB
          Charlie Helin
        2. HDFS-9110.01.patch
          2 kB
          Charlie Helin
        3. HDFS-9110.02.patch
          2 kB
          Charlie Helin
        4. HDFS-9110.03.patch
          2 kB
          Charlie Helin
        5. HDFS-9110.04.patch
          3 kB
          Charlie Helin
        6. HDFS-9110.05.patch
          3 kB
          Charlie Helin
        7. HDFS-9110.06.patch
          3 kB
          Charlie Helin
        8. HDFS-9110.07.patch
          3 kB
          Charlie Helin
        9. HDFS-9110.08.patch
          3 kB
          Charlie Helin

          Activity

            People

            • Assignee:
              chelin Charlie Helin
              Reporter:
              chelin Charlie Helin
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: