[HBASE-28177] mobStore directory do not set storage policy like normal store directory

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 2.5.6
• Component/s: mob
• Labels: pull-request-available
• Assignee: xieyupei
• Reporter: xieyupei
• Due: 25/Oct/23
• Created: 25/Oct/23 16:39
• Updated: 25/Nov/24 06:41

Description

We set block storage policy for store directory only in the HStore, but mobStore path is generate in the HMobStore. I write a test case to describe this bug in TestHRegionFileSystem.

@Test
public void testMobStoreStoragePolicy() throws Exception {
  TEST_UTIL = new HBaseTestingUtil();
  Configuration conf = TEST_UTIL.getConfiguration();
  TEST_UTIL.startMiniCluster();
  Table table = TEST_UTIL.createTable(TABLE_NAME, FAMILIES);
  assertEquals("Should start with empty table", 0, TEST_UTIL.countRows(table));
  HRegionFileSystem regionFs = getHRegionFS(TEST_UTIL.getConnection(), table, conf);
  try (Admin admin = TEST_UTIL.getConnection().getAdmin()) {
    ColumnFamilyDescriptorBuilder cfdA = ColumnFamilyDescriptorBuilder.newBuilder(FAMILIES[0]);
    cfdA.setValue(HStore.BLOCK_STORAGE_POLICY_KEY, "ONE_SSD");
    cfdA.setMobEnabled(true);
    cfdA.setMobThreshold(2L);
    admin.modifyColumnFamily(TABLE_NAME, cfdA.build());
    while (
        TEST_UTIL.getMiniHBaseCluster().getMaster().getAssignmentManager().getRegionStates()
            .hasRegionsInTransition()
    ) {
      Thread.sleep(200);
      LOG.debug("Waiting on table to finish schema altering");
    }

    // flush memstore snapshot into 3 files
    for (long i = 0; i < 3; i++) {
      Put put = new Put(Bytes.toBytes(i));
      put.addColumn(FAMILIES[0], Bytes.toBytes(i), Bytes.toBytes(i));
      put.addColumn(FAMILIES[0], Bytes.toBytes(i + "qf"), Bytes.toBytes(i + "value"));
      table.put(put);
      admin.flush(TABLE_NAME);
    }
    // there should be 3 files in store dir
    FileSystem fs = TEST_UTIL.getDFSCluster().getFileSystem();
    Path storePath = regionFs.getStoreDir(Bytes.toString(FAMILIES[0]));
    Path mobStorePath = MobUtils.getMobFamilyPath(conf, TABLE_NAME, Bytes.toString(FAMILIES[0]));

    FileStatus[] storeFiles = CommonFSUtils.listStatus(fs, storePath);
    FileStatus[] mobStoreFiles = CommonFSUtils.listStatus(fs, mobStorePath);
    assertNotNull(storeFiles);
    assertEquals(3, storeFiles.length);
    assertNotNull(mobStoreFiles);
    assertEquals(3, mobStoreFiles.length);

    for (FileStatus status : storeFiles) {
      assertEquals("ONE_SSD",
          ((HFileSystem) regionFs.getFileSystem()).getStoragePolicyName(status.getPath()));
    }
    for (FileStatus status : mobStoreFiles) {
      assertEquals("ONE_SSD",
          ((HFileSystem) regionFs.getFileSystem()).getStoragePolicyName(status.getPath()));
    }
  } finally {
    table.close();
    TEST_UTIL.deleteTable(TABLE_NAME);
    TEST_UTIL.shutdownMiniCluster();
  }
}

Also we can get storage policy in shell like this:

root@hbase-master:/usr/local/hadoop# ./bin/hdfs storagepolicies -getStoragePolicy -path /hbase/data/default/member/a645c7c2b31371449331a4e4106b073b/info
The storage policy of /hbase/data/default/member/a645c7c2b31371449331a4e4106b073b/info:
BlockStoragePolicy{HOT:7, storageTypes=[DISK], creationFallbacks=[], replicationFallbacks=[ARCHIVE]}
root@hbase-master:/usr/local/hadoop# ./bin/hdfs storagepolicies -getStoragePolicy -path /hbase/mobdir/data/default/member/288b5f8af920a8190cc07bad277debb5/info
The storage policy of /hbase/mobdir/data/default/member/288b5f8af920a8190cc07bad277debb5/info is unspecified 

Issue Links

• links to: GitHub Pull Request #6385