Hadoop Common
  1. Hadoop Common
  2. HADOOP-5618

Convert Storage.storageDirs into a map.

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None

      Description

      Storage.storageDirs is currently declared as an ArrayList. Recent changes made storageDirs a searchable collection. In order to reflect this changes the storageDirs type should be changed to a searchable collection. This will simplify and optimize current code.

        Activity

        Hide
        Jakob Homan added a comment -

        This actually isn't a good idea at this time. The problem is that the implementation detail that storageDirs is an ordered list leaked out into many other places in the code and is now relied on. Many places expect the iterator over the list to traverse it in the order the storage dirs were added to it, not by the path name associated with the storage (the most reasonable key to a map). For instance, in FSEditLog.java, divertFileStreams is expecting to find corresponding storage dirs and edit streams to work over. Of the 27 or so times that the DirIterator is used (which is based on this ArrayList), only one is actually searching for something like a pathname.
        There's certainly room to correct this but it should be done as a larger clean up of the fsimage code. Thoughts?

        Show
        Jakob Homan added a comment - This actually isn't a good idea at this time. The problem is that the implementation detail that storageDirs is an ordered list leaked out into many other places in the code and is now relied on. Many places expect the iterator over the list to traverse it in the order the storage dirs were added to it, not by the path name associated with the storage (the most reasonable key to a map). For instance, in FSEditLog.java, divertFileStreams is expecting to find corresponding storage dirs and edit streams to work over. Of the 27 or so times that the DirIterator is used (which is based on this ArrayList), only one is actually searching for something like a pathname. There's certainly room to correct this but it should be done as a larger clean up of the fsimage code. Thoughts?
        Hide
        Konstantin Shvachko added a comment -

        I agree.

        Show
        Konstantin Shvachko added a comment - I agree.
        Hide
        Jakob Homan added a comment -

        Resolving as won't fix.

        Show
        Jakob Homan added a comment - Resolving as won't fix.

          People

          • Assignee:
            Jakob Homan
            Reporter:
            Konstantin Shvachko
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development