After a new(or a rejoining) JournalNode joins the cluster and starts accepting streamed transactions, it may not have all the previously finalized edit segments in its local storage directory.

To synchronize this JournalNode (J1) with other active JournalNode, this JournalNode needs to copy the finalized edit segments from another JournalNode (J2) which has all the finalized edit segments.

The remote-reader is used by J1 to get edit segments from J2. The local-reader is used by J2 to read edit segment from its local storage directory.

The current plan is to use http server to transfer finalized edit log segments.

This patch includes the following changes:
1. create a http server for journal node
2. have a new get image severlet to be used only by journal node
3. have a test to test file copy between journal nodes

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12522598/HDFS-3201.HDFS-3092.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test files.

-1 patch. The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2274//console

This message is automatically generated.

Split this JIRA into two for better issue management:
HDFS-3283 is used for creating JN http server
and this JIRA is to enable the edit log transfer.

rebased the patch with the branch head

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523051/HDFS-3201.HDFS-3092.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test files.

-1 patch. The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2293//console

This message is automatically generated.

-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12523278/HDFS-3201.HDFS-3092.patch
against trunk revision .

+1 @author. The patch does not contain any @author tags.

+1 tests included. The patch appears to include 1 new or modified test files.

-1 patch. The patch command could not apply the patch.

Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/2303//console

This message is automatically generated.

Hi Brandon, you don't have to submit the patch since it is not for trunk.

• You are actually using DFS_JOURNAL_USER_NAME_KEY in the code, please update the TODO.

  +  // TODO: journal service should use DFS_JOURNAL_USER_NAME_KEY when security is
  +  // enabled
  

• Please remove Journal.getStorageInfo()

• Please use JOURNAL_ATTRIBUTE_KEY

  +          httpServer.setAttribute("localjournal", localJournal);
  

• Move editsToDownload(..) to the test since it is only used there.

The new patch addressed Nicholas' comments. Thanks!

+1 patch looks good.

I have committed this.