Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9110

Use Files.walkFileTree in NNUpgradeUtil#doPreUpgrade for better efficiency

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • 2.7.0
    • 2.8.0, 3.0.0-alpha1
    • None
    • None

    Description

      This is a request to do some cosmetic improvements on top of HDFS-8480. There a couple of File -> java.nio.file.Path conversions which is a little bit distracting.

      The second aspect is more around efficiency, to be perfectly honest I'm not sure what the number of files that may be processed. However as HDFS-8480 eludes to it appears that this number could be significantly large.

      The current implementation is basically a collect and process where all files first is being examined; put into a collection and after that processed. HDFS-8480 could simply be further enhanced by employing a single iteration without creating an intermediary collection of filenames by using a FileWalker

      Attachments

        1. HDFS-9110.00.patch
          2 kB
          Charlie Helin
        2. HDFS-9110.01.patch
          2 kB
          Charlie Helin
        3. HDFS-9110.02.patch
          2 kB
          Charlie Helin
        4. HDFS-9110.03.patch
          2 kB
          Charlie Helin
        5. HDFS-9110.04.patch
          3 kB
          Charlie Helin
        6. HDFS-9110.05.patch
          3 kB
          Charlie Helin
        7. HDFS-9110.06.patch
          3 kB
          Charlie Helin
        8. HDFS-9110.07.patch
          3 kB
          Charlie Helin
        9. HDFS-9110.08.patch
          3 kB
          Charlie Helin

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            chelin Charlie Helin
            chelin Charlie Helin
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment