Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-12499

dfs.namenode.shared.edits.dir property is currently namenode specific key

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Reopened
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: qjm
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      HDFS + Federation cluster +QJM

      dfs.shared.edits.dir property can be set as
      1. dfs.shared.edits.dir.<<nameserviceId>>
      2. dfs.shared.edits.dir.<<nameserviceId>> .<<namenodId>>

      Configuring both ways are supported currently. Option 2 should not be supported, as for a particular nameservice quorum of journal nodes should be same.

      If option 2 is supported, users can configure for a nameservice Id which is having two namenodes, they can configure different values for journal nodes. which is incorrect.

      Example:
      <property>
      <name>dfs.nameservices</name>
      <value>ns1,ns2</value>
      </property>
      <property>
      <name>dfs.ha.namenodes.ns1</name>
      <value>nn1,nn2</value>
      </property>
      <property>
      <name>dfs.ha.namenodes.ns2</name>
      <value>nn1,nn2</value>
      </property>
      <property>
      <name>dfs.namenode.shared.edits.dir.ns1.nn1</name>
      <value>qjournal://mycluster-node-1:8485;mycluster-node-2:8485;mycluster-node-3:8485/ns1</value>
      </property>
      <property>
      <name>dfs.namenode.shared.edits.dir.ns1.nn1</name>
      <value>qjournal://mycluster-node-3:8485;mycluster-node-4:8485;mycluster-node-5:8485/ns1</value>
      </property>
      <property>
      <name>dfs.namenode.shared.edits.dir.ns2.nn1</name>
      <value>qjournal://mycluster-node-1:8485;mycluster-node-2:8485;mycluster-node-3:8485/ns2</value>
      </property>
      <property>
      <name>dfs.namenode.shared.edits.dir.ns2.nn1</name>
      <value>qjournal://mycluster-node-3:8485;mycluster-node-4:8485;mycluster-node-5:8485/ns2</value>
      </property>

      This jira is to discuss do we need to support 2nd option way of configuring or remove it?

        Attachments

        1. HDFS-12499.02.patch
          2 kB
          Bharat Viswanadham
        2. HDFS-12499.01.patch
          2 kB
          Bharat Viswanadham

          Activity

            People

            • Assignee:
              bharat Bharat Viswanadham
              Reporter:
              bharat Bharat Viswanadham
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated: