.META. file ends in '.META' and not '.META.'? Can't it end in .log? xxxxxx.log.META looks a bit odd and I'm sure will trip up someone else trying to find WALs for whatever reason.
Hmm.. "xxxx.log.META" sounds better..
How does splitMetaLog work? It is passed a server name and if we find logs in the filesystem, then we'll split its logs?
Yes.. (much like how it's currently done for the non-meta logs)
What if .META. is currently assigned? The splitting will be to no avail?
Can you please explain when this can happen (that a RS hosting the .META. crashes, and before we do the log splitting, the .META. gets assigned to someone else)? But in general, the behavior shouldn't change in the patch from what the behavior currently is...
When would we ever log this message:
LOG.info("No logs to split");
This is the same log that gets emitted for the general case (in the current codebase). Look at splitLog method in MasterFileSystem. Seems like when the logdir (.logs) doesn't exist, getLogDirs returns an empty list.
Should that be .META. logs to split?
Methods usually have a space between them rather than:
Ack. I missed adding a space. I usually always add spaces without fail (smile)
Why emit two log lines when you could have one that varies with whether meta or not:
This was mostly for my debugging. I'll update this..
Should be a isMetaWAL method? to do the looking for META_LOG_FILE_EXTN... since used in a few places.
We maybe should have done it before but was a bit tougher since had to do the log splitting first... could have broke up the process more ..
I went that route and backtracked.. The problem is that when a RS crashes, we will probably have that RS hosting many regions (including the catalog ones). To recover we have to call process() methods of both SSH and MSSH as a pair. That seemed harder to maintain.. So I kept the existing semantics of one SSH process doing the work for both META and non-meta regions. As I have noted in a previous comment, I have explicitly not handled
ROOT since the root region is going to disappear post HBASE-3171.
Is this needed?
Yeah, since the process() method handles both .META. and non-meta regions. My earlier comment explains why I am not very inclined to separate the handling in process(). But I'll reconsider this part and see if I can do something better.
MetaServices Interface doesn't seem right? Its all about a WAL. You wouldn't have to add new methods? Especially public ones?
Yeah, I shouldn't have made that a public class, etc. and maybe called it MetaWALServices or something. Would doing that alleviate this concern?
When we lose the .META. region, does the RS clean up its .META. log?
I need to check on this.
I don't like extending the RegionServerServices because makes it harder mocking up an RSS... more stuff to add in.
The new methods could just return null (and that's what I have done in the existing mock classes). But I'll consider this.
Should it be MetaFSHLog rather than FSLog w/ boolean args in constructor? Just wondering.
You mean another class.. Seems overkill to me..
We have to add this method?
public static HLog createMetaHLog(final FileSystem fs, final Path root, final String logName,
Nothing in say the log name that would clue us its a .META. file? Could we preface the logName with .META. or something and then we don't have to pass flags to FSHLog? Could have static isMetaWAL method that looks at log name and can tell it .META.?
The "logname" (and this is there in the current trunk) is kind of misleading. It actually signifies the directory name (and maybe I should update the code to call it logdir everywhere)... So in that sense, there is nothing indicative of "meta". Also, I'd prefer passing explicit booleans rather than string based conditionals..
Is this used:
private static final Pattern pattern = Pattern.compile(".
+ private static final Pattern pattern = Pattern.compile(".