Bulk Operation

  1. Choose Issues
  2. Choose Operation
  3. Operation Details
  4. Confirmation

Step 1 of 4: Choose Issues

Cancel

T Patch Info Key Summary Assignee Reporter P Status Resolution Created Updated Due Development
Sub-task PIG-5332

PIG-4059 Implement auto parallelism for pig on spark

Unassigned liyunzhang Major Open Unresolved  
Sub-task PIG-5241

PIG-4059 Specify the hdfs path directly to spark and avoid the unnecessary download and upload in SparkLauncher.java

Nándor Kollár liyunzhang Major Open Unresolved  
Sub-task PIG-5240

PIG-4059 Fix TestPigRunner#simpleMultiQueryTest3 in spark mode for wrong inputStats

Unassigned liyunzhang Major Open Unresolved  
Sub-task PIG-5239

PIG-4059 Investigate why there are duplicated A[3,4] inTestLocationInPhysicalPlan#test in spark mode

Unassigned liyunzhang Major Open Unresolved  
Sub-task PIG-5205

PIG-4059 Duplicate record key info in GlobalRearrangeConverter#ToGroupKeyValueFunction

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-5199

PIG-4059 exclude jline in spark dependency

Ádám Szita liyunzhang Major Closed Fixed  
Sub-task PIG-5197

PIG-4059 Replace IndexedKey with PigNullableWritable in spark branch

Unassigned liyunzhang Major Resolved Won't Fix  
Sub-task PIG-5195

PIG-4059 Upgrade spark to 2.0

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-5192

PIG-4059 Remove schema tuple reference overhead for replicate join hashmap in POFRJoinSpark

Unassigned liyunzhang Major Open Unresolved  
Sub-task PIG-5133

PIG-4059 Commit changes from last round of review on rb

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4920

PIG-4059 Fail to use Javascript UDF in spark yarn client mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4899

PIG-4059 The number of records of input file is calculated wrongly in spark mode in multiquery case

Ádám Szita liyunzhang Major Closed Fixed  
Sub-task PIG-4898

PIG-4059 Fix unit test failure after PIG-4771's patch was checked in

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4891

PIG-4059 Implement FR join by broadcasting small rdd not making more copys of data

Nándor Kollár liyunzhang Major Closed Fixed  
Sub-task PIG-4886

PIG-4059 Add PigSplit#getLocationInfo to fix the NPE found in log in spark mode

liyunzhang liyunzhang Major Resolved Duplicate  
Sub-task PIG-4876

PIG-4059 OutputConsumeIterator can't handle the last buffered tuples for some Operators

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4859

PIG-4059 Need upgrade snappy-java.version to 1.1.1.3

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4857

PIG-4059 Last record is missing in STREAM operator

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4855

PIG-4059 Merge trunk[4] into spark branch

Pallavi Rao Pallavi Rao Major Closed Fixed  
Sub-task PIG-4848

PIG-4059 pig.noSplitCombination=true should always be set internally for a merge join

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4842

PIG-4059 Collected group doesn't work in some cases

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4820

PIG-4059 Merge trunk[3] into spark branch

Pallavi Rao Pallavi Rao Major Closed Fixed  
Sub-task PIG-4809

PIG-4059 Implement to collect metric data like getSMMSpillCount() in SparkJobStats

Unassigned liyunzhang Major Open Unresolved  
Sub-task PIG-4788

PIG-4059 the value BytesRead metric info always returns 0 even the length of input file is not 0 in spark engine

liyunzhang liyunzhang Major Resolved Duplicate  
Sub-task PIG-4784

PIG-4059 Enable "pig.disable.counter“ for spark engine

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4783

PIG-4059 Refactor SparkLauncher for spark engine

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4766

PIG-4059 Ensure GroupBy is optimized for all algebraic Operations

Pallavi Rao Pallavi Rao Major Closed Fixed  
Sub-task PIG-4754

PIG-4059 Fix UT failures in TestScriptLanguage

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4746

PIG-4059 Ensure spark can be run as PIG action in Oozie

Prateek Vaishnav Pallavi Rao Major Open Unresolved  
Sub-task PIG-4741

PIG-4059 the value of $SPARK_DIST_CLASSPATH in pig file is invalid

liyunzhang liyunzhang Major Resolved Not A Problem  
Sub-task PIG-4720

PIG-4059 Spark related JARs are not included when importing project via IDE

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4711

PIG-4059 Tests in TestCombiner fail due to missing leveldb dependency

Pallavi Rao Pallavi Rao Blocker Closed Fixed  
Sub-task PIG-4709

PIG-4059 Support combine for spark mode

Pallavi Rao Pallavi Rao Major Closed Fixed  
Sub-task PIG-4698

PIG-4059 Enable dynamic resource allocation/de-allocation on Yarn backends

Srikanth Sundarrajan Srikanth Sundarrajan Major Closed Fixed  
Sub-task PIG-4693

PIG-4059 Class conflicts: Kryo bundled in spark vs kryo bundled with pig

Srikanth Sundarrajan Srikanth Sundarrajan Major Closed Fixed  
Sub-task PIG-4681

PIG-4059 Enable Pig on Spark to run on Yarn Cluster mode

Srikanth Sundarrajan Srikanth Sundarrajan Major Resolved Won't Fix  
Sub-task PIG-4675

PIG-4059 Operators with multiple predecessors fail under multiquery optimization

liyunzhang Peter Lin Major Closed Fixed  
Sub-task PIG-4667

PIG-4059 Enable Pig on Spark to run on Yarn Client mode

Srikanth Sundarrajan Srikanth Sundarrajan Major Closed Fixed  
Sub-task PIG-4661

PIG-4059 Fix UT failures in TestPigServerLocal

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4660

PIG-4059 Add Spark Unit Tests for SparkPigStats

Xianda Ke Xianda Ke Major Open Unresolved  
Sub-task PIG-4659

PIG-4059 Fix unit test failures in org.apache.pig.test.TestScriptLanguageJavaScript

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4655

PIG-4059 Support InputStats in spark mode

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4645

PIG-4059 Support hadoop-like Counter using spark accumulator

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4634

PIG-4059 Fix records count issues in output statistics

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4633

PIG-4059 Update hadoop version to enable Spark output statistics

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4622

PIG-4059 Skip TestCubeOperator.testIllustrate and TestMultiQueryLocal.testMultiQueryWithIllustrate

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4621

PIG-4059 Enable Illustrate in spark

Jakov Rabinovits liyunzhang Major In Progress Unresolved  
Sub-task PIG-4619

PIG-4059 Cleanup: change the indent size of some files of pig on spark project from 2 to 4 space

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4616

PIG-4059 Fix UT errors of TestPigRunner in Spark mode

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4615

PIG-4059 Fix null keys join in SkewedJoin in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4614

PIG-4059 Enable "TestLocationInPhysicalPlan" in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4613

PIG-4059 Fix unit test failures about TestAssert

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4611

PIG-4059 Fix remaining unit test failures about "TestHBaseStorage" in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4610

PIG-4059 Enable "TestOrcStorage“ unit test in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4607

PIG-4059 Enable "TestRank1","TestRank3" unit tests in spark mode

Xianda Ke liyunzhang Major Closed Fixed  
Sub-task PIG-4606

PIG-4059 Enable "TestDefaultDateTimeZone" unit tests in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4605

PIG-4059 fix a bug when coping Jar to SparkJob working directory

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4604

PIG-4059 Clean up: refactor the package import order in the files under pig/src/org/apache/pig/backend/hadoop/executionengine/spark according to certain rule

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4601

PIG-4059 Implement Merge CoGroup for Spark engine

liyunzhang Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4597

PIG-4059 Enable "TestNullConstant" unit test in spark mode

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4596

PIG-4059 Fix unit test failures about MergeJoinConverter in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4595

PIG-4059 Fix unit test failures about TestFRJoinNullValue in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4594

PIG-4059 Enable "TestMultiQuery" in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4593

PIG-4059 Enable "TestMultiQueryLocal" in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4589

PIG-4059 Fix unit test failure in TestCase

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4588

PIG-4059 Move tests under 'test-spark' target

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4586

PIG-4059 Cleanup: Rename POConverter to RDDConverter

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4585

PIG-4059 Use newAPIHadoopRDD instead of newAPIHadoopFile

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4582

PIG-4059 Enable "TestPruneColumn" in spark mode

Xianda Ke Xianda Ke Major Closed Fixed  
Sub-task PIG-4577

PIG-4059 Use "cogroup" spark api to implement "groupby+secondarysort" case in GlobalRearrangeConverter.java

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4575

PIG-4059 Pass value to MR Partitioners in Spark engine

Mohit Sabharwal Mohit Sabharwal Major Open Unresolved  
Sub-task PIG-4568

PIG-4059 Fix unit test failure in TestSecondarySortSpark

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4565

PIG-4059 Support custom MR partitioners for Spark engine

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4558

PIG-4059 Modify the test.output value from "no" to "yes" to show more error message

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4557

PIG-4059 Fix POGlobalRearrangeSpark copy constructor for Spark engine

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4549

PIG-4059 Set CROSS operation parallelism for Spark engine

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4542

PIG-4059 OutputConsumerIterator should flush buffered records

Mohit Sabharwal Mohit Sabharwal Major Resolved Pending Closed  
Sub-task PIG-4540

PIG-4059 Remove repetitive org.apache.pig.test.Util#isSparkExecType

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4522

PIG-4059 Remove unnecessary store and load when POSplit is encounted

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4518

PIG-4059 SparkOperator should correspond to complete Spark job

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4504

PIG-4059 Enable Secondary key sort feature in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4489

PIG-4059 Enable local mode tests for Spark engine

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task
Patch Available
PIG-4470

PIG-4059 Add apache license header to all spark package source files

Praveen Rachabattuni Praveen Rachabattuni Major Closed Fixed  
Sub-task
Patch Available
PIG-4469

PIG-4059 Remove redundant code, comments in SparkLauncher

Praveen Rachabattuni Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4456

PIG-4059 Sort the leaves by SparkOperator.operatorKey in SparkLauncher#sparkOperToRDD

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4438

PIG-4059 Limit after sort does not work in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4425

PIG-4059 Upgrade to Spark 1.3

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4422

PIG-4059 Implement MergeJoin (as regular join) for Spark engine

Mohit Sabharwal liyunzhang Major Closed Fixed  
Sub-task PIG-4421

PIG-4059 implement visitSkewedJoin in SparkCompiler

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4413

PIG-4059 change from "SparkLauncher#physicalToRDD" to "SparkLauncher#sparkPlanToRDD" after using spark plan in SparkLauncher

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4398

PIG-4059 Merge from trunk (2) [Spark Branch]

Praveen Rachabattuni Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4396

PIG-4059 Move to Spark 1.2

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4393

PIG-4059 Add stats and error reporting for Spark

Mohit Sabharwal Mohit Sabharwal Major Closed Fixed  
Sub-task PIG-4390

PIG-4059 Fix the NPE of System.getenv("SPARK_MASTER") in SparkLauncher.java

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4374

PIG-4059 Add SparkPlan in spark package

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4364

PIG-4059 remove unnessary MR plan code generated in SparkLauncher.java

liyunzhang liyunzhang Major Resolved Not A Problem  
Sub-task PIG-4362

PIG-4059 Make ship work with spark

liyunzhang liyunzhang Major Closed Fixed  
Sub-task
Patch Available
PIG-4346

PIG-4059 Merge from trunk (1) [Spark Branch]

Praveen Rachabattuni Praveen Rachabattuni Major Closed Fixed  
Sub-task
Patch Available
PIG-4323

PIG-4059 PackageConverter hanging in Spark

Carlos Balduz Carlos Balduz Major Patch Available Unresolved  
Sub-task
Patch Available
PIG-4313

PIG-4059 StackOverflowError in LIMIT operation on Spark

Carlos Balduz Carlos Balduz Major Patch Available Unresolved  
Sub-task PIG-4239

PIG-4059 "pig.output.lazy" not works in spark mode

liyunzhang liyunzhang Major Closed Fixed  
Sub-task
Patch Available
PIG-4237

PIG-4059 Error when there is a bag inside an RDD

Carlos Balduz Carlos Balduz Critical Closed Fixed  
Sub-task PIG-4236

PIG-4059 Avoid packaging spark specific jars into pig fat jar

Unassigned Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4234

PIG-4059 Order By error after Group By in Spark

Unassigned Carlos Balduz Major Closed Fixed  
Sub-task PIG-4233

PIG-4059 Package pig along with dependencies into a fat jar while job submission to Spark cluster

Praveen Rachabattuni Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4232

PIG-4059 UDFContext is not initialized in executors when running on Spark cluster

liyunzhang Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4231

PIG-4059 Make rank work with Spark

Carlos Balduz Carlos Balduz Major Closed Fixed  
Sub-task PIG-4229

PIG-4059 Copy spark dependencies to lib directory

Praveen Rachabattuni Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4228

PIG-4059 SchemaTupleBackend error when working on a Spark 1.1.0 cluster

Unassigned Carlos Balduz Major Open Unresolved  
Sub-task PIG-4209

PIG-4059 Make stream work with Spark

liyunzhang Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4208

PIG-4059 Make merge-sparse join work with Spark

Abhishek Agarwal Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4207

PIG-4059 Make python udfs work with Spark

liyunzhang Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4206

PIG-4059 e2e tests for Spark can not work in hadoop env

liyunzhang liyunzhang Major Closed Fixed  
Sub-task PIG-4200

PIG-4059 Make merge join work with Spark engine

Praveen Rachabattuni Praveen Rachabattuni Major Resolved Duplicate  
Sub-task PIG-4193

PIG-4059 Make collected group work with Spark

Praveen Rachabattuni Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4192

PIG-4059 Make ruby udfs work with Spark

liyunzhang Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4191

PIG-4059 Make skewed join work with Spark

Praveen Rachabattuni Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4190

PIG-4059 Implement replicated join in Spark engine

Mohit Sabharwal Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4189

PIG-4059 Make cross join work with Spark

Mohit Sabharwal Praveen Rachabattuni Major Resolved Duplicate  
Sub-task PIG-4183

PIG-4059 Fix classpath error when using pig command with Spark

liyunzhang liyunzhang Major Resolved Done  
Sub-task PIG-4174

PIG-4059 e2e tests for Spark

Praveen Rachabattuni Praveen Rachabattuni Major Closed Fixed  
Sub-task PIG-4173

PIG-4059 Move to Spark 1.x

Richard Ding bc Wong Major Closed Fixed  
Sub-task PIG-4168

PIG-4059 Initial implementation of unit tests for Pig on Spark

liyunzhang Praveen Rachabattuni Major Closed Fixed  
Sub-task
Patch Available
PIG-4167

PIG-4059 Initial implementation of Pig on Spark

Praveen Rachabattuni Praveen Rachabattuni Major Closed Fixed  

Cancel