Details
-
Task
-
Status: Reopened
-
Major
-
Resolution: Unresolved
-
None
-
None
Attachments
Attachments
- AMBARI-19702.1.patch
- 11 kB
- Daniel Dai
- AMBARI-19702.2.patch
- 11 kB
- Daniel Dai
- AMBARI-19702.3.patch
- 2 kB
- Daniel Dai
- AMBARI-19702.4.patch
- 3 kB
- Daniel Dai
Activity
Pushed to trunk, commit 2f06c1cefac30e0a53d631cf2a2a6aef64a89e0c
branch-2.5, commit 84c680a9ef14581463dc333b36ebcd3603cdaa78
FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #849 (See https://builds.apache.org/job/Ambari-branch-2.5/849/)
AMBARI-19702. Switch Pig default engine to tez in HDP 2.6 (Daniel Dai (afernandez: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=84c680a9ef14581463dc333b36ebcd3603cdaa78)
- (add) ambari-server/src/main/resources/stacks/HDP/2.6/services/PIG/configuration/pig-properties.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml
FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #6599 (See https://builds.apache.org/job/Ambari-trunk-Commit/6599/)
AMBARI-19702. Switch Pig default engine to tez in HDP 2.6 (Daniel Dai (afernandez: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=2f06c1cefac30e0a53d631cf2a2a6aef64a89e0c)
- (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml
- (add) ambari-server/src/main/resources/stacks/HDP/2.6/services/PIG/configuration/pig-properties.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/config-upgrade.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/config-upgrade.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/nonrolling-upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/config-upgrade.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/nonrolling-upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/nonrolling-upgrade-2.6.xml
daijy, afernandez, this prevents Ambari Server start, and also causes unit test failures:
ERROR [main] ModuleFileUnmarshaller:141 - Cannot parse /var/lib/ambari-server/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml ERROR [main] ModuleFileUnmarshaller:143 - Upgrade pack must contain pre-downgrade elements if pre-upgrade exists for processing component PIG/PIG
Could you please check?
Sorry about this. Attach AMBARI-19702.3.patch. Can you check if this fixes the issue? Or if you want me to create a separate Jira, let me know.
daijy, yes, AMBARI-19702.3.patch fixes the issues. Thanks for the quick response.
afernandez, could you please commit AMBARI-19702.3.patch, too, to resolve this issue? Thanks.
daijy, I pushed addendum AMBARI-19702.3.patch to
trunk, commit 772c093073b31fb0887d7072622e4c37c227e28d
branch-2.5, commit 4116c4092d1e32f685dbf379f9ca2ec89718d89d
FAILURE: Integrated in Jenkins build Ambari-branch-2.5 #856 (See https://builds.apache.org/job/Ambari-branch-2.5/856/)
AMBARI-19702. ADDENDUM. Switch Pig default engine to tez in HDP 2.6 (afernandez: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=4116c4092d1e32f685dbf379f9ca2ec89718d89d)
- (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml
FAILURE: Integrated in Jenkins build Ambari-trunk-Commit #6605 (See https://builds.apache.org/job/Ambari-trunk-Commit/6605/)
AMBARI-19702. ADDENDUM. Switch Pig default engine to tez in HDP 2.6 (afernandez: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=772c093073b31fb0887d7072622e4c37c227e28d)
- (edit) ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.4/upgrades/upgrade-2.6.xml
- (edit) ambari-server/src/main/resources/stacks/HDP/2.3/upgrades/upgrade-2.6.xml
I just saw the review for this and I believe the code is incorrect. You wanted to change a value inside of pig-properties/content:
# exectype local|mapreduce|tez, mapreduce is default
exectype=mapreduce
However, the upgrade work you did specifies:
<service name="PIG"> <component name="PIG"> <changes> <definition xsi:type="configure" id="hdp_2_6_0_0_pig_use_tez"> <type>pig-properties</type> <replace key="exectype" find="exectype=mapreduce" replace-with="exectype=tez" /> </definition> </changes> </component> </service>
The key is not exectype ... the key would need to be content since this is a part of pig-properties/content ...
cc afernandez since you committed this on behalf of the author...
Thanks jonathanhurley, attach AMBARI-19702.4.patch to fix it. Also change the comment "mapreduce is default" to "tez is default" in the patch.
avijayan, afernandez, can you help commit to 2.5.0 branch? Otherwise, we leave it in a half done status.
-1. Fix it and ship it.
I think you meant to edit ambari-server/src/main/resources/stacks/HDP/2.5/upgrades/upgrade-2.6.xml instead of ambari-server/src/main/resources/stacks/HDP/2.6/upgrades/upgrade-2.6.xml
Within the same stack version, we don't apply config changes.
Everything else looks good, thanks for making this change.
cc daijy