Pig
  1. Pig
  2. PIG-4100

TEZ bug broke hadoop compilation 2 in trunk

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Not a Problem
    • Affects Version/s: None
    • Fix Version/s: 0.14.0
    • Component/s: tez
    • Labels:
      None

      Description

      Cannot compile pig trunk (with parameter -Dhadoopversion=23) due to this error

      [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java:594: error: cannot find symbol
      [javac] vertex.setLocationHint(new VertexLocationHint(tezOp.getLoaderInfo().getInputSplitInfo().getTaskLocationHints()));
      [javac] ^
      [javac] symbol: method setLocationHint(VertexLocationHint)
      [javac] location: variable vertex of type Vertex

        Activity

        Hide
        Quang-Nhat HOANG-XUAN added a comment -

        Compiled perfectly!
        Thank you.

        Show
        Quang-Nhat HOANG-XUAN added a comment - Compiled perfectly! Thank you.
        Hide
        Cheolsoo Park added a comment -

        Quang-Nhat HOANG-XUAN, that looks like your workspace issue. I can build cleanly, and the apache jenkin builds also succeeds-
        https://builds.apache.org/job/Pig-trunk/1623/changes

        Can you try remove ~/.ivy2/cache/org.apache.tez and ~/.m2/repository/org/apache/tez and rebuild?

        Show
        Cheolsoo Park added a comment - Quang-Nhat HOANG-XUAN , that looks like your workspace issue. I can build cleanly, and the apache jenkin builds also succeeds- https://builds.apache.org/job/Pig-trunk/1623/changes Can you try remove ~/.ivy2/cache/org.apache.tez and ~/.m2/repository/org/apache/tez and rebuild?
        Hide
        Quang-Nhat HOANG-XUAN added a comment -

        Hi, After checking out, I tried to build by "ant -Dhadoopversion=23" and still got these errors:

        [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java:594: error: cannot find symbol
        [javac] vertex.setLocationHint(new VertexLocationHint(tezOp.getLoaderInfo().getInputSplitInfo().getTaskLocationHints()));
        [javac] ^
        [javac] symbol: method setLocationHint(VertexLocationHint)
        [javac] location: variable vertex of type Vertex
        [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java:80: error: cannot find symbol
        [javac] pc.getProperties().setProperty(TezRuntimeConfiguration.TEZ_RUNTIME_OPTIMIZE_LOCAL_FETCH, "true");
        [javac] ^
        [javac] symbol: variable TEZ_RUNTIME_OPTIMIZE_LOCAL_FETCH
        [javac] location: class TezRuntimeConfiguration
        [javac] Note: Some input files use or override a deprecated API.
        [javac] Note: Recompile with -Xlint:deprecation for details.
        [javac] Note: Some input files use unchecked or unsafe operations.
        [javac] Note: Recompile with -Xlint:unchecked for details.
        [javac] 2 errors
        [javac] 3 warnings

        Show
        Quang-Nhat HOANG-XUAN added a comment - Hi, After checking out, I tried to build by "ant -Dhadoopversion=23" and still got these errors: [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezDagBuilder.java:594: error: cannot find symbol [javac] vertex.setLocationHint(new VertexLocationHint(tezOp.getLoaderInfo().getInputSplitInfo().getTaskLocationHints())); [javac] ^ [javac] symbol: method setLocationHint(VertexLocationHint) [javac] location: variable vertex of type Vertex [javac] /home/hoang/DATA/trunk-test/trunk/src/org/apache/pig/backend/hadoop/executionengine/tez/TezLauncher.java:80: error: cannot find symbol [javac] pc.getProperties().setProperty(TezRuntimeConfiguration.TEZ_RUNTIME_OPTIMIZE_LOCAL_FETCH, "true"); [javac] ^ [javac] symbol: variable TEZ_RUNTIME_OPTIMIZE_LOCAL_FETCH [javac] location: class TezRuntimeConfiguration [javac] Note: Some input files use or override a deprecated API. [javac] Note: Recompile with -Xlint:deprecation for details. [javac] Note: Some input files use unchecked or unsafe operations. [javac] Note: Recompile with -Xlint:unchecked for details. [javac] 2 errors [javac] 3 warnings
        Hide
        Cheolsoo Park added a comment -

        Closing the jira now.

        Show
        Cheolsoo Park added a comment - Closing the jira now.
        Hide
        Daniel Dai added a comment -

        SNAPSHOT published.

        Show
        Daniel Dai added a comment - SNAPSHOT published.
        Hide
        Cheolsoo Park added a comment -

        If it's delayed, I can revert PIG-4098 for now. Let me know.

        Show
        Cheolsoo Park added a comment - If it's delayed, I can revert PIG-4098 for now. Let me know.
        Hide
        Daniel Dai added a comment -

        Let me ask Tez folks to publish a SNAPSHOT.

        Show
        Daniel Dai added a comment - Let me ask Tez folks to publish a SNAPSHOT.
        Hide
        Cheolsoo Park added a comment -

        Indeed, my bad.

        Show
        Cheolsoo Park added a comment - Indeed, my bad.
        Hide
        Koji Noguchi added a comment -

        Tez snapshot jars are not published to maven yet, so you're seeing an error.

        We should have waited for this to happen before committing PIG-4098?

        Show
        Koji Noguchi added a comment - Tez snapshot jars are not published to maven yet, so you're seeing an error. We should have waited for this to happen before committing PIG-4098 ?
        Hide
        Quang-Nhat HOANG-XUAN added a comment -

        I got it.
        Thank you.

        Show
        Quang-Nhat HOANG-XUAN added a comment - I got it. Thank you.
        Hide
        Cheolsoo Park added a comment - - edited

        Sorry for the inconvenience. This is because of PIG-4098.

        Tez snapshot jars are not published to maven yet, so you're seeing an error. But it will be resolved as soon as Tez jenkin build run. One way to workaround is to manually run mvn install -DskipTests with Tez trunk.

        Show
        Cheolsoo Park added a comment - - edited Sorry for the inconvenience. This is because of PIG-4098 . Tez snapshot jars are not published to maven yet, so you're seeing an error. But it will be resolved as soon as Tez jenkin build run. One way to workaround is to manually run mvn install -DskipTests with Tez trunk .

          People

          • Assignee:
            Unassigned
            Reporter:
            Quang-Nhat HOANG-XUAN
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development