[PIG-3861] duplicate jars get added to distributed cache - ASF JIRA

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Bug
Status: Closed
Priority: Minor
Resolution: Fixed
Affects Version/s: None
Fix Version/s: 0.14.0
Component/s: None
Labels:
None

Hadoop Flags:

Reviewed

Description

PigContext's scriptJars should handle de-duplication of jars to account for script engines e.g. JythonScriptEngine performing various jar loading for module and sometimes adding same jar twice. AlsoJobControlCompiler.shipToHdfs() needs a check against adding the same jar more than once, under different randomly incremented sub-dirs.

Attachments

Attachments

- Sort By Name
- Sort By Date
- Ascending
- Descending

PIG-3861-fixtest.patch
26/Oct/14 03:27
0.7 kB
Daniel Dai
PIG-3861-5.patch
21/Oct/14 22:07
12 kB
Rohini Palaniswamy
PIG-3861-4.patch
29/May/14 20:36
22 kB
Mona Chitnis
PIG-3861-3.patch
28/Apr/14 15:48
12 kB
Mona Chitnis
PIG-3861-2.patch
15/Apr/14 00:34
13 kB
Mona Chitnis
PIG-3681-1.patch
03/Apr/14 00:13
7 kB
Mona Chitnis

Issue Links

Add Link

is related to

PIG-4373 Implement PIG-3861 in Tez

Patch Available

Delete this link

requires

PIG-3798 registered jar in pig script are appended to the classpath multiple times

Closed

Delete this link

Activity

Comment

This comment will be Viewable by All Users Viewable by All Users

Cancel

People

Assignee:: Mona Chitnis

Reporter:: Mona Chitnis

Votes:: 0 Vote for this issue

Watchers:: 3 Start watching this issue

Dates

Created:: 03/Apr/14 00:08

Updated:: 09/Jul/18 22:09

Resolved:: 23/Oct/14 20:49

Agile

Slack

Issue deployment