Uploaded image for project: 'Ambari'
  1. Ambari
  2. AMBARI-23766

Corrupt mapreduce/tez tar.gz may be uploaded to HDFS if parallel execution is enabled

    XMLWordPrintableJSON

    Details

      Description

      If parallel_execution on Ambari Agent is enabled, two components (History Server and Hive Server) may create and upload the MapReduce and Tez archives concurrently. This could result in a corrupt uploaded file, preventing Hive Server from starting.

      output-32.txt
      2018-05-04 07:24:40,421 - Creating a new Tez tarball at /var/lib/ambari-agent/tmp/tez-native-tarball-staging/tez-native.tar.gz
      ...
      2018-05-04 07:24:49,751 - Creating new file /hdp/apps/2.6.4.5-2/tez/tez.tar.gz in DFS
      2018-05-04 07:24:49,753 - call['ambari-sudo.sh su hdfs -l -s /bin/bash -c 'curl -sS -L -w '"'"'%{http_code}'"'"' -X PUT --data-binary @/var/lib/ambari-agent/tmp/tez-native-tarball-staging/tez-native.tar.gz -H '"'"'Content-Type: application/octet-stream'"'"' '"'"'http://localhost:50070/webhdfs/v1/hdp/apps/2.6.4.5-2/tez/tez.tar.gz?op=CREATE&user.name=hdfs&overwrite=True&permission=444'"'"' 1>/tmp/tmpoEINHo 2>/tmp/tmpkjlNxP''] {'logoutput': None, 'quiet': False}
      2018-05-04 07:24:51,629 - call returned (0, '')
      
      output-30.txt
      2018-05-04 07:24:46,818 - Creating a new Tez tarball at /var/lib/ambari-agent/tmp/tez-native-tarball-staging/tez-native.tar.gz
      ...
      2018-05-04 07:24:54,117 - DFS file /hdp/apps/2.6.4.5-2/tez/tez.tar.gz is identical to /var/lib/ambari-agent/tmp/tez-native-tarball-staging/tez-native.tar.gz, skipping the copying
      

        Attachments

          Activity

            People

            • Assignee:
              adoroszlai Attila Doroszlai
              Reporter:
              adoroszlai Attila Doroszlai
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 3h
                3h