[HDDS-4289] Throw exception from hadoop2 filesystem jar in HA environment

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Workaround
• Fix Version/s: 1.1.0
• Component/s: OM HA
• Labels: newbie pull-request-available
• Assignee: Janus Chow
• Reporter: Marton Elek
• Created: 29/Sep/20 09:28
• Updated: 24/Feb/21 08:13
• Resolved: 24/Feb/21 08:13

Description

Thanks for Tamas Pleszkan for reporting this problem.

ozone-filesystem-hadoop2 doesn't support OM-HA (today) as the used Hadoop3OmTransport uses FailoverProxyProvider which is not available in hadoop2.

Long-term we need a custom failover mechanism, but this jira suggests to improve the error handling. `Hadoop27OmTransportFactory` should throw an exception if HA is used.

Used command:

spark-submit --master yarn --deploy-mode client --executor-memory 1g --conf "spark.yarn.access.hadoopFileSystems=o3fs://bucket.hdfs.ozone1/" --jars "/opt/cloudera/parcels/CDH-7.1.3-1.cdh7.1.3.p0.4992530/jars/hadoop-ozone-filesystem-hadoop2-0.5.0.7.1.3.0-100.jar" SparkWordCount.py o3fs://bucket.hdfs.ozone1/words 2

Current exception:

Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.ozone.om.exceptions.OMNotLeaderException): OM:om2 is not the leader. Suggested leader is OM:om1.

Expected exception: Unsupported operation exception with meaningful hint to use hadoop3 filesystem jar.

Issue Links

• is superceded by: HDDS-4764 Add HA-OM support for ozonefs-hadoop2
• links to: GitHub Pull Request #1776