Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-2383

[puppet] ruby template to generate yarn config for standalone and ha is wrong.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.1.0
    • Fix Version/s: 1.1.0
    • Component/s: deployment
    • Labels:
      None

      Description

      As per hue.ini config following configs needs to be present for rm ha

      [[yarn_clusters]]
      
          [[[default]]]
            # Enter the host on which you are running the ResourceManager
            ## resourcemanager_host=localhost
      
            # The port where the ResourceManager IPC listens on
            ## resourcemanager_port=8032
      
            # Whether to submit jobs to this cluster
            submit_to=True
      
            # Resource Manager logical name (required for HA)
            ## logical_name=
      
            # Change this if your YARN cluster is Kerberos-secured
            ## security_enabled=false
      
            # URL of the ResourceManager API
            ## resourcemanager_api_url=http://localhost:8088
      
            # URL of the ProxyServer API
            ## proxy_api_url=http://localhost:8088
      
            # URL of the HistoryServer API
            ## history_server_api_url=http://localhost:19888
      
            # URL of the Spark History Server
            ## spark_history_server_url=http://localhost:18088
      
            # In secure mode (HTTPS), if SSL certificates from YARN Rest APIs
            # have to be verified against certificate authority
            ## ssl_cert_ca_verify=True
      
          # HA support by specifying multiple clusters.
          # Redefine different properties there.
          # e.g.
      
          # [[[ha]]]
            # Resource Manager logical name (required for HA)
            ## logical_name=my-rm-name
      
            # Un-comment to enable
            ## submit_to=True
      
            # URL of the ResourceManager API
            ## resourcemanager_api_url=http://localhost:8088
      
            # ...
      
      

      Where as hue template does not have [[[ha]]] and [[[default]] tag. also for few config

        [[yarn_clusters]]
      <% resourcemanager_hosts = Array(@rm_host) -%>
      <% resourcemanager_hosts.each do |host| -%>
        [[[<%= host %>]]]
            # Enter the host on which you are running the ResourceManager
            resourcemanager_host=<%= host %>
      
            # The port where the ResourceManager IPC listens on
            # resourcemanager_port=<% rm_port %>
      
            # Whether to submit jobs to this cluster
            submit_to=True
      <%     if @rm_logical_name -%>
            # Resource Manager logical name (required for HA)
            logical_name=<%= @rm_logical_name %>
      <%     else -%>
            # Resource Manager logical name (required for HA)
            # logical_name=
      <%     end -%>
      
            # Change this if your YARN cluster is Kerberos-secured
            security_enabled=<%= if (@kerberos_realm != "") ; "true" else "false" end %>
      
            # URL of the ResourceManager API
            resourcemanager_api_url=http://<%= host %>:<%= @rm_api_port %>
      
            # URL of the ProxyServer API
            proxy_api_url=<%= @rm_proxy_url %>
      
            # URL of the HistoryServer API
            history_server_api_url=<%= @history_server_url %>
      
            # URL of the NodeManager API
            node_manager_api_url=http://localhost:8042
      <% end -%>
      
      

        Attachments

        1. BIGTOP-2383.patch
          3 kB
          Vishal Khandelwal

          Activity

            People

            • Assignee:
              vishk Vishal Khandelwal
              Reporter:
              vishk Vishal Khandelwal
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: