Uploaded image for project: 'Accumulo'
  1. Accumulo
  2. ACCUMULO-118

accumulo could work across HDFS instances, which would help it to scale past a single namenode

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: master, tserver
    • Labels:
      None

      Description

      Consider using full path names to files, which would allow the servers to access the files on any HDFS file system.

      Work may exist elsewhere to run HDFS using a number of NameNode instances to break up the namespace.

      We may need a pluggable strategy to determine namespace for new files.

        Attachments

        1. ACCUMULO-118-02.txt
          6 kB
          Eric Newton
        2. ACCUMULO-118-01.txt
          5 kB
          Eric Newton

          Issue Links

          1.
          create a utility for converting the !METADATA table entry to a full path Sub-task Resolved Unassigned  
          2.
          ServerConstants.getBaseDirs() use fs default name, not instance.dfs.uri Sub-task Resolved Keith Turner  
          3.
          Get disk usage will not work across namenodes Sub-task Resolved Eric Newton  
          4.
          Offline map reduce will fail if tablet spans multiple namenodes Sub-task Resolved Keith Turner  
          5.
          RFile printinfo command does not handle multiple namenodes Sub-task Resolved Keith Turner  
          6.
          Monitor collects disk usage from single namenode Sub-task Resolved Josh Elser  
          7.
          Bulk import does sanity checks on client side using a single filesystem Sub-task Resolved Keith Turner  
          8.
          Changing Accumulo config can prevent locating root table files. Sub-task Resolved Keith Turner  
          9.
          Analyze all usages of instance.dfs.uri and instance.volumes Sub-task Resolved Keith Turner  
          10.
          Instance id and version info only stored on one hdfs instance Sub-task Resolved Keith Turner  
          11.
          Need utility to decommission dfs uris Sub-task Resolved Keith Turner  
          12.
          Garbage collector may delete referenced files after upgrade Sub-task Resolved Keith Turner  
          13.
          Write ahead logs from upgrade prematurely GCed Sub-task Resolved Eric Newton  
          14.
          Create utility for rewriting uris Sub-task Resolved Keith Turner

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 0.5h
          15.
          WALog entries not properly deleted after recovery during upgrade 1.5 -> 1.6 Sub-task Resolved Eric Newton  
          16.
          Accumulo fails when instance.volumes and fs.default.name are disjoint Sub-task Resolved Keith Turner  
          17.
          Recommend using viewfs:// or HA Namenode Sub-task Resolved Keith Turner  
          18.
          Deprecate instance.dfs.uri and instance.dfs.dir Sub-task Resolved Josh Elser  
          19.
          general.volume.chooser has no environment Sub-task Resolved Eric Newton  
          20.
          Log recovery is converting paths to relative Sub-task Resolved Keith Turner  
          21.
          updateAccumuloVersion only updates the data version on a single configured volume Sub-task Resolved Josh Elser  
          22.
          Copying failed bulk imports seems broken Sub-task Resolved Eric Newton  
          23.
          Suppress expected deprecation warnings from instance.dfs.{dir,uri} Sub-task Resolved Josh Elser  
          24.
          FileUtil expects instance.dfs.dir in tmpDir path Sub-task Resolved Josh Elser  
          25.
          Update ClientOpts to read from volumes or instance dir Sub-task Resolved Josh Elser  

            Activity

              People

              • Assignee:
                ecn Eric Newton
                Reporter:
                ecn Eric Newton
              • Votes:
                3 Vote for this issue
                Watchers:
                13 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - 2,016h Original Estimate - 2,016h
                  2,016h
                  Remaining:
                  Remaining Estimate - 2,016h
                  2,016h
                  Logged:
                  Remaining Estimate - 2,016h
                  0.5h