Details
-
Improvement
-
Status: Resolved
-
Critical
-
Resolution: Invalid
-
0.21.0
Description
I've tried to do HDFS-RAID over hadoop-0.21.0 ...
But, It's failed to raid when I've tried to start raidnode (start-raidnode.sh)
What do I have to correct it?
This is the information. Would you tell me how to correct it?
Namenode : xxx.23.129.40:9006
Datanode: xxx.23.129.39, 38
Raidnode : xxx.23.129.40 ( I want to seperate raidnoe from namenode, how can I do?)
- hdfs-site.xml
<name>hdfs.raid.locations</name>
<value>hdfs://xxx.23.129.40:9006/raid</value>
=> (What's the meaning of this, which address do I have to write down?)
<description>The location for parity files. If this is
is not defined, then defaults to /raid.
</description>
</property>
- raid.xml
<configuration>
<srcPath prefix="hdfs://xxx.23.129.40:9006/TEST">
=> (What's the meaning of this, which address do I have to write down?)
<policy name = "TEST"> => (What's this? )
<property>
<name>srcReplication</name>
<value>3</value>
<description> pick files for RAID only if their replication factor is
greater than or equal to this value.
</description>
</property>
<property>
<name>targetReplication</name>
<value>2</value>
<description> after RAIDing, decrease the replication factor of a file to
this value.
</description>
</property>
<property>
<name>metaReplication</name>
<value>2</value>
<description> the replication factor of the RAID meta file
</description>
</property>
<property>
<name>modTimePeriod</name>
<value>100</value>
<description> time (milliseconds) after a file is modified to make it a
candidate for RAIDing
</description>
</property>
*log: when start-raidnode.sh
/************************************************************
SHUTDOWN_MSG: Shutting down RaidNode at test40/xxx.23.129.40
************************************************************/
2011-11-25 23:59:13,417 INFO org.apache.hadoop.raid.RaidNode: STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting RaidNode
STARTUP_MSG: host = test40/xxx.23.129.40
STARTUP_MSG: args = []
STARTUP_MSG: version = 0.21.0
************************************************************/
2011-11-25 23:59:13,518 ERROR org.apache.hadoop.raid.ConfigManager: Reloading config
file /KTBHOME/hadoop/hadoop/conf/raid.xml
2011-11-25 23:59:13,611 INFO org.apache.hadoop.security.Groups: Group mapping
impl=org.apache.hadoop.security.ShellBasedUnixGroupsMapping; cacheTimeout=300000
2011-11-25 23:59:13,647 WARN org.apache.hadoop.conf.Configuration: mapred.task.id is
deprecated. Instead, use mapreduce.task.attempt.id
2011-11-25 23:59:13,714 INFO org.apache.hadoop.raid.ConfigManager:
TEST.srcReplication = 3
2011-11-25 23:59:13,714 INFO org.apache.hadoop.raid.ConfigManager:
TEST.targetReplication = 2
2011-11-25 23:59:13,715 INFO org.apache.hadoop.raid.ConfigManager:
TEST.metaReplication = 2
2011-11-25 23:59:13,715 INFO org.apache.hadoop.raid.ConfigManager:
TEST.modTimePeriod = 100
2011-11-25 23:59:13,716 INFO org.apache.hadoop.raid.ConfigManager:
k_table1.targetReplication = 1
2011-11-25 23:59:13,716 INFO org.apache.hadoop.raid.ConfigManager:
k_table1.metaReplication = 2
2011-11-25 23:59:13,716 INFO org.apache.hadoop.raid.ConfigManager:
k_table1.modTimePeriod = 100
2011-11-25 23:59:13,728 INFO org.apache.hadoop.ipc.Server: Starting SocketReader
2011-11-25 23:59:13,751 INFO org.apache.hadoop.ipc.metrics.RpcMetrics: Initializing
RPC Metrics with hostName=RaidNode, port=60000
2011-11-25 23:59:13,810 INFO org.apache.hadoop.ipc.metrics.RpcDetailedMetrics:
Initializing RPC Metrics with hostName=RaidNode, port=60000
2011-11-25 23:59:13,812 INFO org.apache.hadoop.raid.RaidNode: RaidNode up at:
/127.0.0.1:60000
2011-11-25 23:59:13,813 INFO org.apache.hadoop.ipc.Server: IPC Server Responder:
starting
2011-11-25 23:59:13,813 INFO org.apache.hadoop.ipc.Server: IPC Server listener on
60000: starting
2011-11-25 23:59:13,814 INFO org.apache.hadoop.ipc.Server: IPC Server handler 0 on
60000: starting
2011-11-25 23:59:13,814 INFO org.apache.hadoop.ipc.Server: IPC Server handler 1 on
60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 2 on
60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 3 on
60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 4 on
60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 5 on
60000: starting
2011-11-25 23:59:13,815 INFO org.apache.hadoop.ipc.Server: IPC Server handler 6 on
60000: starting
2011-11-25 23:59:13,816 INFO org.apache.hadoop.ipc.Server: IPC Server handler 7 on
60000: starting
2011-11-25 23:59:13,816 INFO org.apache.hadoop.ipc.Server: IPC Server handler 8 on
60000: starting
2011-11-25 23:59:13,816 INFO org.apache.hadoop.ipc.Server: IPC Server handler 9 on
60000: starting
2011-11-25 23:59:13,818 INFO org.apache.hadoop.raid.RaidNode: Triggering Policy
Filter TEST hdfs://xxx.23.129.40:9006/TEST
2011-11-25 23:59:13,819 INFO org.apache.hadoop.raid.RaidNode: Started archive scan
2011-11-25 23:59:13,828 INFO org.apache.hadoop.raid.RaidNode: No filtered paths for
policy TEST
2011-11-25 23:59:13,828 INFO org.apache.hadoop.raid.RaidNode: Triggering Policy
Filter k_table1 hdfs://xxx.23.129.40:9006
2011-11-25 23:59:13,829 INFO org.apache.hadoop.raid.RaidNode: No filtered paths for
policy k_table1