Pig
  1. Pig
  2. PIG-3724

pig e2e tests dont have hadoop libs on classpath

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Won't Fix
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Patch Info:
      Patch Available

      Description

      The following command throws errors in test/e2e/pig/udfs/java/build.xml
      because the hadoop classpaths are not found.

      ant -Dharness.old.pig=~/pig-0.11
      -Dharness.cluster.conf=/opt/mapr/hadoop/hadoop-0.20.2/conf
      -Dharness.cluster.bin=/usr/bin/hadoop
      -Dharness.hadoop.home=/opt/mapr/hadoop/hadoop-0.20.2 test-e2e

      I have uploaded a patch which includes lib directories from build/ivy/lib/Pig to solve this problem.

      1. PIG-3724.patch
        0.8 kB
        Suhas Satish

        Activity

        Hide
        Rohini Palaniswamy added a comment -

        Suhas,
        The hadoop libs should be picked up from harness.hadoop.home and should not be including anything from ivy/lib as that is only for compile time and should not be used for runtime. If that is not the case you need to see why they are not picked up and if your hadoop home does not have the hadoop jars in the expected location.

        Show
        Rohini Palaniswamy added a comment - Suhas, The hadoop libs should be picked up from harness.hadoop.home and should not be including anything from ivy/lib as that is only for compile time and should not be used for runtime. If that is not the case you need to see why they are not picked up and if your hadoop home does not have the hadoop jars in the expected location.
        Hide
        Cheolsoo Park added a comment -

        Rohini is right. I have no problem with running e2e tests with EMR Hadoop. I have no knowledge about MapR distribution, but can you please check whether this is your environment issue? If so, please mark the patch as "won't fix".

        Canceling patch.

        Show
        Cheolsoo Park added a comment - Rohini is right. I have no problem with running e2e tests with EMR Hadoop. I have no knowledge about MapR distribution, but can you please check whether this is your environment issue? If so, please mark the patch as "won't fix". Canceling patch.
        Hide
        Suhas Satish added a comment -

        marked as won't fix. I agree with Rohini

        Show
        Suhas Satish added a comment - marked as won't fix. I agree with Rohini

          People

          • Assignee:
            Suhas Satish
            Reporter:
            Suhas Satish
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development