Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-8712

HFileLinkCleaner (FSUtils.listStatus) logs too much if links do not exists

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.95.1
    • Fix Version/s: 0.98.0, 0.95.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Testing 0.95.1 RC1, I found that just inserting data and splitting seems to trigger a bug in the archivedHFileCleaner that makes it complain about missing files. It seems to happen mostly to reference files post-split.

      I'm attaching a grep I did with the file name (you'll see the same file name happens to be in 2 regions, it's because it's a reference file).

      1. archivedHFileCleaner-errors.log
        12 kB
        Jean-Daniel Cryans
      2. HBASE-8712-v0.patch
        0.8 kB
        Matteo Bertozzi

        Activity

        Hide
        jdcryans Jean-Daniel Cryans added a comment -

        Attaching the log.

        Show
        jdcryans Jean-Daniel Cryans added a comment - Attaching the log.
        Hide
        jdcryans Jean-Daniel Cryans added a comment -

        Assigning to Matteo, he said it's a newb error and it should not even print.

        Show
        jdcryans Jean-Daniel Cryans added a comment - Assigning to Matteo, he said it's a newb error and it should not even print.
        Hide
        mbertozzi Matteo Bertozzi added a comment -

        This is not a race or a bug... is just that FSUtils.listStatus() is too verbose if the directory doesn't exists...
        In this case is the HFileLinkCleaner (enabled with snapshot) that tries to look for cloned tables by looking inside the .link-xyz directory... and there're no cloned tables (and no .link-xyz directory)... so you get the DEBUG log...

        Show
        mbertozzi Matteo Bertozzi added a comment - This is not a race or a bug... is just that FSUtils.listStatus() is too verbose if the directory doesn't exists... In this case is the HFileLinkCleaner (enabled with snapshot) that tries to look for cloned tables by looking inside the .link-xyz directory... and there're no cloned tables (and no .link-xyz directory)... so you get the DEBUG log...
        Hide
        jdcryans Jean-Daniel Cryans added a comment -

        I kinda wish that we could do something better than that, TRACEing or DEBUGing a missing file is kinda strange isn't it? Why is it missing? Just like the code that's printing the exception, we're missing context.

        Anyways, I'll commit this. Thanks Matteo.

        Show
        jdcryans Jean-Daniel Cryans added a comment - I kinda wish that we could do something better than that, TRACEing or DEBUGing a missing file is kinda strange isn't it? Why is it missing? Just like the code that's printing the exception, we're missing context. Anyways, I'll commit this. Thanks Matteo.
        Hide
        jdcryans Jean-Daniel Cryans added a comment -

        Committed to branch and trunk, thanks Matteo.

        Show
        jdcryans Jean-Daniel Cryans added a comment - Committed to branch and trunk, thanks Matteo.
        Hide
        hudson Hudson added a comment -

        Integrated in HBase-TRUNK #4165 (See https://builds.apache.org/job/HBase-TRUNK/4165/)
        HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if
        links do not exists (Matteo via JD) (Revision 1490871)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Show
        hudson Hudson added a comment - Integrated in HBase-TRUNK #4165 (See https://builds.apache.org/job/HBase-TRUNK/4165/ ) HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if links do not exists (Matteo via JD) (Revision 1490871) Result = FAILURE jdcryans : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Hide
        hudson Hudson added a comment -

        Integrated in hbase-0.95 #230 (See https://builds.apache.org/job/hbase-0.95/230/)
        HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if
        links do not exists (Matteo via JD) (Revision 1490869)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Show
        hudson Hudson added a comment - Integrated in hbase-0.95 #230 (See https://builds.apache.org/job/hbase-0.95/230/ ) HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if links do not exists (Matteo via JD) (Revision 1490869) Result = FAILURE jdcryans : Files : /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Hide
        hudson Hudson added a comment -

        Integrated in hbase-0.95-on-hadoop2 #125 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/125/)
        HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if
        links do not exists (Matteo via JD) (Revision 1490869)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Show
        hudson Hudson added a comment - Integrated in hbase-0.95-on-hadoop2 #125 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/125/ ) HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if links do not exists (Matteo via JD) (Revision 1490869) Result = FAILURE jdcryans : Files : /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Hide
        hudson Hudson added a comment -

        Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #559 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/559/)
        HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if
        links do not exists (Matteo via JD) (Revision 1490871)

        Result = FAILURE
        jdcryans :
        Files :

        • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java
        Show
        hudson Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #559 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/559/ ) HBASE-8712 HFileLinkCleaner (FSUtils.listStatus) logs too much if links do not exists (Matteo via JD) (Revision 1490871) Result = FAILURE jdcryans : Files : /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/util/FSUtils.java

          People

          • Assignee:
            mbertozzi Matteo Bertozzi
            Reporter:
            jdcryans Jean-Daniel Cryans
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development