Hadoop Common
  1. Hadoop Common
  2. HADOOP-9489

Eclipse instructions in BUILDING.txt don't work

    Details

    • Type: Bug Bug
    • Status: Patch Available
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: 2.7.0
    • Fix Version/s: None
    • Component/s: build
    • Labels:
    • Target Version/s:

      Description

      I have tried several times to import Hadoop trunk into Eclipse following the instructions in the BUILDING.txt file, but so far have not been able to get it to work.

      If I use a fresh install of Eclipse 4.2.2, Eclipse will complain about an undefined M2_REPO environment variable. I discovered that this is defined automatically by the M2Eclipse plugin, and think that the BUILDING.txt doc should be updated to explain this.

      After installing M2Eclipse I tried importing the code again, and now get over 2500 errors related to missing class dependencies. Many of these errors correspond to missing classes in the oah*.proto namespace, which makes me think that 'mvn eclipse:eclipse' is not triggering protoc.

      1. eclipse_hadoop_errors.txt
        451 kB
        Carl Steinbach
      2. error.log
        15 kB
        Tsuyoshi Ozawa
      3. HADOOP-9489.003.patch
        3 kB
        Chris Nauroth
      4. HADOOP-9489.004.patch
        2 kB
        Chris Nauroth
      5. HADOOP-9489.1.patch
        3 kB
        Chris Nauroth
      6. HADOOP-9489.2.patch
        3 kB
        Chris Nauroth

        Activity

        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Open Open Patch Available Patch Available
        573d 14h 33m 1 Chris Nauroth 14/Nov/14 22:25
        Hide
        Ravi Prakash added a comment -

        Also I import as Existing Eclipse Projects (Rather than using the m2eclipse plugin) . With m2eclipse I find that it complains with "Plugin execution not covered by lifecycle configuration: org.apache.hadoop:hadoop-maven-plugins:3.0.0-SNAPSHOT:protoc (execution: compile-protoc, phase: generate-sources)"

        Show
        Ravi Prakash added a comment - Also I import as Existing Eclipse Projects (Rather than using the m2eclipse plugin) . With m2eclipse I find that it complains with "Plugin execution not covered by lifecycle configuration: org.apache.hadoop:hadoop-maven-plugins:3.0.0-SNAPSHOT:protoc (execution: compile-protoc, phase: generate-sources)"
        Hide
        Ravi Prakash added a comment -

        Thanks for the patches Chris! I use Eclipse and I am able to import the projects without this patch. I do build the project before running mvn eclipse:eclipse, so I'm a +1 to adding that in BUILDING.txt. We should probably also level expectations (some newcomers may expect to be able to build and "run" the project from Eclipse; I've never been able to do this) I find Eclipse immensely useful for navigating the code, attaching remote debuggers and stepping through, but I build and run only on the command line using mvn . I wonder if that is other peoples' process too?

        Show
        Ravi Prakash added a comment - Thanks for the patches Chris! I use Eclipse and I am able to import the projects without this patch. I do build the project before running mvn eclipse:eclipse, so I'm a +1 to adding that in BUILDING.txt. We should probably also level expectations (some newcomers may expect to be able to build and "run" the project from Eclipse; I've never been able to do this) I find Eclipse immensely useful for navigating the code, attaching remote debuggers and stepping through, but I build and run only on the command line using mvn . I wonder if that is other peoples' process too?
        Allen Wittenauer made changes -
        Labels BB2015-05-RFC BB2015-05-TBR
        Hide
        Hadoop QA added a comment -



        -1 overall



        Vote Subsystem Runtime Comment
        0 pre-patch 14m 40s Pre-patch trunk compilation is healthy.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        +1 javac 7m 29s There were no new javac warning messages.
        +1 javadoc 9m 33s There were no new javadoc warning messages.
        +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
        +1 whitespace 0m 1s The patch has no lines that end in whitespace.
        +1 install 1m 39s mvn install still works.
        +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse.
        +1 common tests 22m 30s Tests passed in hadoop-common.
            56m 52s  



        Subsystem Report/Notes
        Patch URL http://issues.apache.org/jira/secure/attachment/12701924/HADOOP-9489.004.patch
        Optional Tests javadoc javac unit
        git revision trunk / ed0f4db
        hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6564/artifact/patchprocess/testrun_hadoop-common.txt
        Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6564/testReport/
        Java 1.7.0_55
        uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6564/console

        This message was automatically generated.

        Show
        Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 14m 40s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 7m 29s There were no new javac warning messages. +1 javadoc 9m 33s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. +1 whitespace 0m 1s The patch has no lines that end in whitespace. +1 install 1m 39s mvn install still works. +1 eclipse:eclipse 0m 32s The patch built with eclipse:eclipse. +1 common tests 22m 30s Tests passed in hadoop-common.     56m 52s   Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12701924/HADOOP-9489.004.patch Optional Tests javadoc javac unit git revision trunk / ed0f4db hadoop-common test log https://builds.apache.org/job/PreCommit-HADOOP-Build/6564/artifact/patchprocess/testrun_hadoop-common.txt Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/6564/testReport/ Java 1.7.0_55 uname Linux asf906.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/6564/console This message was automatically generated.
        Ray Chiang made changes -
        Labels BB2015-05-RFC
        Ray Chiang made changes -
        Labels BB2015-05-TBR
        Allen Wittenauer made changes -
        Labels BB2015-05-TBR
        Vinod Kumar Vavilapalli made changes -
        Target Version/s 2.7.0 [ 12327583 ] 2.8.0 [ 12329058 ]
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12701924/HADOOP-9489.004.patch
        against trunk revision ca1c00b.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5814//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5814//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12701924/HADOOP-9489.004.patch against trunk revision ca1c00b. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5814//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5814//console This message is automatically generated.
        Chris Nauroth made changes -
        Attachment HADOOP-9489.004.patch [ 12701924 ]
        Hide
        Chris Nauroth added a comment -

        Thanks, Chengbing Liu. Here is patch v004, without the change in the root pom.xml file.

        Show
        Chris Nauroth added a comment - Thanks, Chengbing Liu . Here is patch v004, without the change in the root pom.xml file.
        Hide
        Chengbing Liu added a comment -

        Chris Nauroth The following description

        m2eclipse didn't know how to handle the goals in our custom Maven plugin for calling protoc and generating version info. I updated the top-level pom.xml to run the protoc goal during an Eclipse build and ignore the version-info goal. This eliminated nearly all of the remaining ~60 errors.

        is not a problem any more. So I think we can just remove the change made to

        {base_dir}

        /pom.xml, keep the rest of it, and test again.

        Show
        Chengbing Liu added a comment - Chris Nauroth The following description m2eclipse didn't know how to handle the goals in our custom Maven plugin for calling protoc and generating version info. I updated the top-level pom.xml to run the protoc goal during an Eclipse build and ignore the version-info goal. This eliminated nearly all of the remaining ~60 errors. is not a problem any more. So I think we can just remove the change made to {base_dir} /pom.xml, keep the rest of it, and test again.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12701728/HADOOP-9489.003.patch
        against trunk revision aa55fd3.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5802//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5802//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12701728/HADOOP-9489.003.patch against trunk revision aa55fd3. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5802//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5802//console This message is automatically generated.
        Chris Nauroth made changes -
        Assignee Chris Nauroth [ cnauroth ]
        Hide
        Chris Nauroth added a comment -

        Thanks, Tsuyoshi. If this isn't working consistently, then I'm inclined to abandon the patch. Is there anyone else who'd want to pick it up from here and try to work through the errors Tsuyoshi saw?

        Show
        Chris Nauroth added a comment - Thanks, Tsuyoshi. If this isn't working consistently, then I'm inclined to abandon the patch. Is there anyone else who'd want to pick it up from here and try to work through the errors Tsuyoshi saw?
        Chris Nauroth made changes -
        Attachment HADOOP-9489.003.patch [ 12701728 ]
        Hide
        Chris Nauroth added a comment -

        Thanks for the reviews. I'm attaching patch v003 with the changed hyperlink as suggested by Tsuyoshi.

        Robert Joseph Evans, you had some earlier comments about this patch:

        https://issues.apache.org/jira/browse/HADOOP-9489?focusedCommentId=13638027&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13638027

        Do you object to committing this patch? It sounds like some people have found it useful to apply locally in their environments.

        I'm not a regular Eclipse user, so I'm going to defer to other community members' judgment on this.

        Show
        Chris Nauroth added a comment - Thanks for the reviews. I'm attaching patch v003 with the changed hyperlink as suggested by Tsuyoshi. Robert Joseph Evans , you had some earlier comments about this patch: https://issues.apache.org/jira/browse/HADOOP-9489?focusedCommentId=13638027&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13638027 Do you object to committing this patch? It sounds like some people have found it useful to apply locally in their environments. I'm not a regular Eclipse user, so I'm going to defer to other community members' judgment on this.
        Tsuyoshi Ozawa made changes -
        Attachment error.log [ 12701727 ]
        Hide
        Tsuyoshi Ozawa added a comment -

        Attaching error log.

        Show
        Tsuyoshi Ozawa added a comment - Attaching error log.
        Hide
        Tsuyoshi Ozawa added a comment -

        I got some errors:

        An internal error occurred during: "Importing Maven projects".
        Missing parameter for pluginExecutionFilter. groupId, artifactId, versionRange and goals must be specificed, but found: groupId = 'org.apache.hadoop'
        artifactId = 'hadoop-maven-plugins'
        versionRange = 'null'
        goals = '[protoc]'

        Additionally, I also got errors about objects generated by probobuf.

        Show
        Tsuyoshi Ozawa added a comment - I got some errors: An internal error occurred during: "Importing Maven projects". Missing parameter for pluginExecutionFilter. groupId, artifactId, versionRange and goals must be specificed, but found: groupId = 'org.apache.hadoop' artifactId = 'hadoop-maven-plugins' versionRange = 'null' goals = ' [protoc] ' Additionally, I also got errors about objects generated by probobuf.
        Hide
        Tsuyoshi Ozawa added a comment -

        Carl Steinbach Thank you for taknig this JIRA!

        http://www.sonatype.org/m2eclipse/
        

        The link should be http://eclipse.org/m2e/ since http://www.sonatype.org/m2eclipse/ is redirected to the unrelated page.

        Show
        Tsuyoshi Ozawa added a comment - Carl Steinbach Thank you for taknig this JIRA! http: //www.sonatype.org/m2eclipse/ The link should be http://eclipse.org/m2e/ since http://www.sonatype.org/m2eclipse/ is redirected to the unrelated page.
        Hide
        Allen Wittenauer added a comment -

        +1

        Show
        Allen Wittenauer added a comment - +1
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12681630/HADOOP-9489.2.patch
        against trunk revision 8ca0d95.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5791//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5791//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12681630/HADOOP-9489.2.patch against trunk revision 8ca0d95. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 core tests . The patch passed unit tests in hadoop-common-project/hadoop-common. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5791//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5791//console This message is automatically generated.
        Hide
        Chengbing Liu added a comment -

        This is quite helpful to Eclipse users, I think we should have it in the next release. Thanks Chris Nauroth!

        Show
        Chengbing Liu added a comment - This is quite helpful to Eclipse users, I think we should have it in the next release. Thanks Chris Nauroth !
        Hide
        Chris Nauroth added a comment -

        The test failure is unrelated. This is a change in build scripts only.

        Show
        Chris Nauroth added a comment - The test failure is unrelated. This is a change in build scripts only.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12681630/HADOOP-9489.2.patch
        against trunk revision 49c3889.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. There were no new javadoc warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

        org.apache.hadoop.security.ssl.TestReloadingX509TrustManager

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5086//testReport/
        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5086//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12681630/HADOOP-9489.2.patch against trunk revision 49c3889. +1 @author . The patch does not contain any @author tags. -1 tests included . The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.security.ssl.TestReloadingX509TrustManager +1 contrib tests . The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/5086//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/5086//console This message is automatically generated.
        Chris Nauroth made changes -
        Affects Version/s 2.7.0 [ 12327583 ]
        Affects Version/s 3.0.0 [ 12320357 ]
        Target Version/s 3.0.0 [ 12320357 ] 2.7.0 [ 12327583 ]
        Priority Major [ 3 ] Minor [ 4 ]
        Chris Nauroth made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Chris Nauroth made changes -
        Attachment HADOOP-9489.2.patch [ 12681630 ]
        Hide
        Chris Nauroth added a comment -

        I just noticed that this ancient patch of mine never went in. The old patch still applied with some warnings, but here is a rebase just to make it cleaner.

        Is this patch still helpful? I'm not a regular Eclipse user, so I'm probably not the best judge.

        Show
        Chris Nauroth added a comment - I just noticed that this ancient patch of mine never went in. The old patch still applied with some warnings, but here is a rebase just to make it cleaner. Is this patch still helpful? I'm not a regular Eclipse user, so I'm probably not the best judge.
        Hide
        Lokesh Basu added a comment -

        Thanks Chris. The patch provided by you helped me build hadoop on my system. I have been trying to build this for two days. Thanks again

        Show
        Lokesh Basu added a comment - Thanks Chris. The patch provided by you helped me build hadoop on my system. I have been trying to build this for two days. Thanks again
        Hide
        Robert Joseph Evans added a comment -

        I am not really sure that we want to require m2eclipse plug-in. It the past using mvn eclipse:eclipse and the m2plugin caused conflicts with one another, although recently there have not been as many issues with that.

        http://maven.apache.org/plugins/maven-eclipse-plugin/

        has instructions on how to use the maven-eclipse-plugin. This is what is run when you run eclipse:eclipse. You can set M2_REPO manually inside eclipse, or you can run mvn eclipse:configure-workspace to set the M2_REPO. Please refer to

        http://maven.apache.org/plugins/maven-eclipse-plugin/configure-workspace-mojo.html

        for more details about that command because you may need to define some other parameters on the command line to make it work properly with your environment.

        Show
        Robert Joseph Evans added a comment - I am not really sure that we want to require m2eclipse plug-in. It the past using mvn eclipse:eclipse and the m2plugin caused conflicts with one another, although recently there have not been as many issues with that. http://maven.apache.org/plugins/maven-eclipse-plugin/ has instructions on how to use the maven-eclipse-plugin. This is what is run when you run eclipse:eclipse. You can set M2_REPO manually inside eclipse, or you can run mvn eclipse:configure-workspace to set the M2_REPO. Please refer to http://maven.apache.org/plugins/maven-eclipse-plugin/configure-workspace-mojo.html for more details about that command because you may need to define some other parameters on the command line to make it work properly with your environment.
        Chris Nauroth made changes -
        Affects Version/s 3.0.0 [ 12320357 ]
        Target Version/s 3.0.0 [ 12320357 ]
        Chris Nauroth made changes -
        Attachment HADOOP-9489.1.patch [ 12579713 ]
        Hide
        Chris Nauroth added a comment -

        Hi, Carl Steinbach. I think I understand what's going wrong with fresh imports of the project, and I'm attaching a patch that attempts to fix it. Would you mind trying out this patch in your environment to see if it works for you?

        Here is a summary of what I did:

        1. I changed BUILDING.txt to state that m2eclipse is required, that you should do an mvn install of the whole Hadoop project (not just hadoop-maven-plugins), and that you should import by clicking File -> Import -> Maven -> Existing Maven Projects. Once I did it this way, a fresh import showed ~100 errors.
        2. hadoop-common still had some errors for missing compiled protoc test sources. This was because the pom.xml was not specifying generated-test-sources/java as a source directory. I added a call to build-helper-maven-plugin to set that up correctly. This eliminated ~40 errors for me.
        3. m2eclipse didn't know how to handle the goals in our custom Maven plugin for calling protoc and generating version info. I updated the top-level pom.xml to run the protoc goal during an Eclipse build and ignore the version-info goal. This eliminated nearly all of the remaining ~60 errors.

        After all of that, a fresh import still gives me one error:

        Project 'hadoop-streaming' is missing required source folder: '/Users/chris/git/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf'

        It appears that Eclipse doesn't like us to have a test resource pointing at a path outside of the module. I'm not yet sure how to fix this one (other than just copying the file we need out of hadoop-yarn-server-resourcemanager into hadoop-streaming, but then we'd have a double-maintenance issue). It doesn't appear to be fatal though.

        I'm curious to see if these fixes address all of the problems that you saw. Thanks!

        Show
        Chris Nauroth added a comment - Hi, Carl Steinbach . I think I understand what's going wrong with fresh imports of the project, and I'm attaching a patch that attempts to fix it. Would you mind trying out this patch in your environment to see if it works for you? Here is a summary of what I did: I changed BUILDING.txt to state that m2eclipse is required, that you should do an mvn install of the whole Hadoop project (not just hadoop-maven-plugins), and that you should import by clicking File -> Import -> Maven -> Existing Maven Projects. Once I did it this way, a fresh import showed ~100 errors. hadoop-common still had some errors for missing compiled protoc test sources. This was because the pom.xml was not specifying generated-test-sources/java as a source directory. I added a call to build-helper-maven-plugin to set that up correctly. This eliminated ~40 errors for me. m2eclipse didn't know how to handle the goals in our custom Maven plugin for calling protoc and generating version info. I updated the top-level pom.xml to run the protoc goal during an Eclipse build and ignore the version-info goal. This eliminated nearly all of the remaining ~60 errors. After all of that, a fresh import still gives me one error: Project 'hadoop-streaming' is missing required source folder: '/Users/chris/git/hadoop-common/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf' It appears that Eclipse doesn't like us to have a test resource pointing at a path outside of the module. I'm not yet sure how to fix this one (other than just copying the file we need out of hadoop-yarn-server-resourcemanager into hadoop-streaming, but then we'd have a double-maintenance issue). It doesn't appear to be fatal though. I'm curious to see if these fixes address all of the problems that you saw. Thanks!
        Chris Nauroth made changes -
        Assignee Chris Nauroth [ cnauroth ]
        Carl Steinbach made changes -
        Field Original Value New Value
        Attachment eclipse_hadoop_errors.txt [ 12579667 ]
        Hide
        Carl Steinbach added a comment -

        Attaching a file containing the list build failures flagged by Eclipse.

        Show
        Carl Steinbach added a comment - Attaching a file containing the list build failures flagged by Eclipse.
        Carl Steinbach created issue -

          People

          • Assignee:
            Unassigned
            Reporter:
            Carl Steinbach
          • Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

            • Created:
              Updated:

              Development