Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-24111

Enable CompactionTool executions on non-HDFS filesystems

    XMLWordPrintableJSON

    Details

      Description

      CompactionTool on non-HDFS filesystems does not work because the used filesystems are mixed up. To collect the StoreFiles the Filesystem.get(conf) method is used instead of relying on the root dir filesystem.

      With YARN the delegation tokens are not handled correctly with a different filesystem and the mappers only get delegation token for the staging directory's filesystem.

      Another issue is in MapReduce mode when YARN is used. In this case the TMP directory (hbase.tmp.dir) is created under yarn.nodemanager.local-dirs but this directory is created (even on HDFS a regular user can't create this directory) on the Storefile's filesystem where a local filesystem temp directory shouldn't be used.

      In this ticket I plan to clean up the used filesystems, remove the custom temp directory (use the regions' .tmp directory) and collect delegation tokens for staging dir + store dir paths' FileSystems.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                psomogyi Peter Somogyi
                Reporter:
                psomogyi Peter Somogyi
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: