HBase
  1. HBase
  2. HBASE-8712

HFileLinkCleaner (FSUtils.listStatus) logs too much if links do not exists

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.95.1
    • Fix Version/s: 0.98.0, 0.95.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Testing 0.95.1 RC1, I found that just inserting data and splitting seems to trigger a bug in the archivedHFileCleaner that makes it complain about missing files. It seems to happen mostly to reference files post-split.

      I'm attaching a grep I did with the file name (you'll see the same file name happens to be in 2 regions, it's because it's a reference file).

      1. archivedHFileCleaner-errors.log
        12 kB
        Jean-Daniel Cryans
      2. HBASE-8712-v0.patch
        0.8 kB
        Matteo Bertozzi

        Activity

        Hide
        Jean-Daniel Cryans added a comment -

        Attaching the log.

        Show
        Jean-Daniel Cryans added a comment - Attaching the log.
        Hide
        Jean-Daniel Cryans added a comment -

        Assigning to Matteo, he said it's a newb error and it should not even print.

        Show
        Jean-Daniel Cryans added a comment - Assigning to Matteo, he said it's a newb error and it should not even print.
        Hide
        Matteo Bertozzi added a comment -

        This is not a race or a bug... is just that FSUtils.listStatus() is too verbose if the directory doesn't exists...
        In this case is the HFileLinkCleaner (enabled with snapshot) that tries to look for cloned tables by looking inside the .link-xyz directory... and there're no cloned tables (and no .link-xyz directory)... so you get the DEBUG log...

        Show
        Matteo Bertozzi added a comment - This is not a race or a bug... is just that FSUtils.listStatus() is too verbose if the directory doesn't exists... In this case is the HFileLinkCleaner (enabled with snapshot) that tries to look for cloned tables by looking inside the .link-xyz directory... and there're no cloned tables (and no .link-xyz directory)... so you get the DEBUG log...
        Hide
        Jean-Daniel Cryans added a comment -

        I kinda wish that we could do something better than that, TRACEing or DEBUGing a missing file is kinda strange isn't it? Why is it missing? Just like the code that's printing the exception, we're missing context.

        Anyways, I'll commit this. Thanks Matteo.

        Show
        Jean-Daniel Cryans added a comment - I kinda wish that we could do something better than that, TRACEing or DEBUGing a missing file is kinda strange isn't it? Why is it missing? Just like the code that's printing the exception, we're missing context. Anyways, I'll commit this. Thanks Matteo.
        Hide
        Jean-Daniel Cryans added a comment -

        Committed to branch and trunk, thanks Matteo.

        Show
        Jean-Daniel Cryans added a comment - Committed to branch and trunk, thanks Matteo.
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK #4165 (See https://builds.apache.org/job/HBase-TRUNK/4165/)
        HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if
        links do not exists (Matteo via JD) (Revision 1490871)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK #4165 (See https://builds.apache.org/job/HBase-TRUNK/4165/ ) HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if links do not exists (Matteo via JD) (Revision 1490871) Result = FAILURE jdcryans : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95 #230 (See https://builds.apache.org/job/hbase-0.95/230/)
        HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if
        links do not exists (Matteo via JD) (Revision 1490869)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Show
        Hudson added a comment - Integrated in hbase-0.95 #230 (See https://builds.apache.org/job/hbase-0.95/230/ ) HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if links do not exists (Matteo via JD) (Revision 1490869) Result = FAILURE jdcryans : Files : /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Hide
        Hudson added a comment -

        Integrated in hbase-0.95-on-hadoop2 #125 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/125/)
        HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if
        links do not exists (Matteo via JD) (Revision 1490869)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Show
        Hudson added a comment - Integrated in hbase-0.95-on-hadoop2 #125 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/125/ ) HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if links do not exists (Matteo via JD) (Revision 1490869) Result = FAILURE jdcryans : Files : /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Hide
        Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #559 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/559/)
        HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if
        links do not exists (Matteo via JD) (Revision 1490871)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Show
        Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #559 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/559/ ) HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if links do not exists (Matteo via JD) (Revision 1490871) Result = FAILURE jdcryans : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java

          People

          • Assignee:
            Matteo Bertozzi
            Reporter:
            Jean-Daniel Cryans
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development