[HBASE-7108] Don't use legal family name for system folder at region level - ASF JIRA

Details

Type: Bug
Status: Patch Available
Priority: Major
Resolution: Unresolved
Affects Version/s: 0.92.2, 0.94.2, 0.95.2
Fix Version/s: 2.0.0
Component/s: regionserver
Labels:
None

Description

CHANGED, was: Don't allow "recovered.edits" as legal family name

Region directories can contain folders called "recovered.edits", log splitting related.
But there's nothing that prevent a user to create a family with that name...

HLog.RECOVERED_EDITS_DIR = "recovered.edits";
HRegion.MERGEDIR = "merges"; // fixed with ~~HBASE-6158~~
SplitTransaction.SPLITDIR = "splits"; // fixed with ~~HBASE-6158~~

Options
- Sort By Name
- Sort By Date
- Ascending
- Descending

Attachments

HBASE-7108-v0.patch

06/Nov/12 22:59

2 kB

Matteo Bertozzi

Issue Links

is related to

HBASE-7640 FSUtils#getTableStoreFilePathMap should ignore non-hfiles.

Resolved

Activity

Ascending order - Click to sort in descending order

Matteo Bertozzi created issue - 06/Nov/12 22:58

Hide

Permalink

Matteo Bertozzi added a comment - 06/Nov/12 22:59

added patch to mark "recovered.edits" as Illegal family name

Show

Matteo Bertozzi added a comment - 06/Nov/12 22:59 added patch to mark "recovered.edits" as Illegal family name

Matteo Bertozzi made changes - 06/Nov/12 22:59

Field	Original Value	New Value
Attachment		HBASE-7108-v0.patch [ 12552359 ]

Matteo Bertozzi made changes - 06/Nov/12 22:59

Status

Open [ 1 ]

Patch Available [ 10002 ]

Hide

Permalink

Todd Lipcon added a comment - 06/Nov/12 22:59

Instead of disallowing it, could we change recovered.edits to be something starting with a '.'? I think we already have some requirement that CFs not start with '.', right? (if not, is there any other prefix which we've already disallowed for users?)

Show

Todd Lipcon added a comment - 06/Nov/12 22:59 Instead of disallowing it, could we change recovered.edits to be something starting with a '.'? I think we already have some requirement that CFs not start with '.', right? (if not, is there any other prefix which we've already disallowed for users?)

Hide

Permalink

Matteo Bertozzi added a comment - 06/Nov/12 23:05

yeah good catch, we don't allow the '.' as starting character in the family name.
So, can I just rename "recovered.edits" in ".recovered.edits"?
do we preserve fs layout compatibility in some way?

Show

Matteo Bertozzi added a comment - 06/Nov/12 23:05 yeah good catch, we don't allow the '.' as starting character in the family name. So, can I just rename "recovered.edits" in ".recovered.edits"? do we preserve fs layout compatibility in some way?

Hide

Permalink

Todd Lipcon added a comment - 07/Nov/12 00:43

For compatibility, one option would be:

in 0.94: change code to recognize both "recovered.edits" and ".recovered.edits" at region-open time
in 0.96: change code to write to ".recovered.edits" and recognize both

Then, so long as someone is upgrading from the most recent 0.94 to 0.96, they'd be fine. An upgrade from an older 0.94 or 0.92 to 0.96 would potentially have an issue if there were a failure in the middle of a rolling upgrade.

Would that be acceptible?

Show

Todd Lipcon added a comment - 07/Nov/12 00:43 For compatibility, one option would be: in 0.94: change code to recognize both "recovered.edits" and ".recovered.edits" at region-open time in 0.96: change code to write to ".recovered.edits" and recognize both Then, so long as someone is upgrading from the most recent 0.94 to 0.96, they'd be fine. An upgrade from an older 0.94 or 0.92 to 0.96 would potentially have an issue if there were a failure in the middle of a rolling upgrade. Would that be acceptible?

Hide

Permalink

Hadoop QA added a comment - 07/Nov/12 01:27

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12552359/HBASE-7108-v0.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.

+1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

-1 javadoc. The javadoc tool appears to have generated 87 warning messages.

+1 javac. The applied patch does not increase the total number of javac compiler warnings.

-1 findbugs. The patch appears to introduce 15 new Findbugs (version 1.3.9) warnings.

+1 release audit. The applied patch does not increase the total number of release audit warnings.

-1 core tests. The patch failed these unit tests:
org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat

This message is automatically generated.

Show

Hadoop QA added a comment - 07/Nov/12 01:27 -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12552359/HBASE-7108-v0.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. -1 javadoc . The javadoc tool appears to have generated 87 warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. -1 findbugs . The patch appears to introduce 15 new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.mapreduce.TestHFileOutputFormat Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/3239//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop2-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/3239//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/3239//console This message is automatically generated.

Matteo Bertozzi made changes - 18/Nov/12 13:13

Summary	Don't allow "recovered.edits" as legal family name	Don't use legal family name for system folder at region level
Description	Region directories can contain a folder called "recovered.edits", log splitting related. But there's nothing that prevent a user to create a family with that name...	CHANGED, was: Don't allow "recovered.edits" as legal family name Region directories can contain folders called "recovered.edits", log splitting related. But there's nothing that prevent a user to create a family with that name... HLog.RECOVERED_EDITS_DIR = "recovered.edits"; HRegion.MERGEDIR = "merges"; // fixed with ~~HBASE-6158~~ SplitTransaction.SPLITDIR = "splits"; // fixed with ~~HBASE-6158~~

Hide

Permalink

stack added a comment - 18/Nov/12 23:02

Todd's suggestion sounds good to me.

Show

stack added a comment - 18/Nov/12 23:02 Todd's suggestion sounds good to me.

Hide

Permalink

stack added a comment - 11/Jan/13 07:46

You up for making the suggested change Matteo Bertozzi? Would be good one to get into 0.96. Good on you.

Show

stack added a comment - 11/Jan/13 07:46 You up for making the suggested change Matteo Bertozzi ? Would be good one to get into 0.96. Good on you.

Jimmy Xiang made changes - 21/Jan/13 22:59

Link

This issue is related to ~~HBASE-7640~~ [ ~~HBASE-7640~~ ]

stack made changes - 26/Feb/13 07:06

Fix Version/s		0.95.0 [ 12324094 ]
Fix Version/s	0.96.0 [ 12320040 ]

stack made changes - 04/Apr/13 01:10

Fix Version/s		0.95.1 [ 12324288 ]
Fix Version/s	0.95.0 [ 12324094 ]

stack made changes - 12/Jun/13 05:48

Fix Version/s		0.95.2 [ 12320040 ]
Fix Version/s	0.95.1 [ 12324288 ]

stack made changes - 15/Aug/13 20:22

Fix Version/s		0.96.0 [ 12324822 ]
Fix Version/s	0.95.2 [ 12320040 ]

stack made changes - 19/Oct/13 20:27

Fix Version/s		0.96.1 [ 12324961 ]
Fix Version/s	0.96.0 [ 12324822 ]

stack made changes - 16/Dec/13 18:42

Fix Version/s		0.99.0 [ 12325675 ]
Fix Version/s	0.96.1 [ 12324961 ]

Hide

Permalink

Hadoop QA added a comment - 16/Dec/13 20:12

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12552359/HBASE-7108-v0.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

-1 patch. The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8180//console

This message is automatically generated.

Show

Hadoop QA added a comment - 16/Dec/13 20:12 -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12552359/HBASE-7108-v0.patch against trunk revision . +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/8180//console This message is automatically generated.

Enis Soztutar made changes - 09/Sep/14 07:13

Fix Version/s		2.0.0 [ 12327188 ]
Fix Version/s	0.99.0 [ 12325675 ]

Hide

Permalink

Hadoop QA added a comment - 09/Sep/14 07:23

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12552359/HBASE-7108-v0.patch
against trunk revision .
ATTACHMENT ID: 12552359

+1 @author. The patch does not contain any @author tags.

-1 patch. The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10780//console

This message is automatically generated.

Show

Hadoop QA added a comment - 09/Sep/14 07:23 -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12552359/HBASE-7108-v0.patch against trunk revision . ATTACHMENT ID: 12552359 +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. -1 patch . The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/10780//console This message is automatically generated.

People

Assignee:

Matteo Bertozzi

Reporter:

Matteo Bertozzi

Votes:

0 Vote for this issue

Watchers:

4 Start watching this issue

Dates

Created:

06/Nov/12 22:58

Updated:

09/Sep/14 07:23

Development

Agile

View on Board