HBase
  1. HBase
  2. HBASE-6145

Fix site target post modularization

    Details

    • Type: Task Task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    1. 6145v4.txt
      2.15 MB
      stack
    2. 6145v4.txt
      2.15 MB
      stack
    3. sitev3.txt
      1.43 MB
      stack
    4. site2.txt
      1.42 MB
      stack
    5. site.txt
      1.35 MB
      stack

      Activity

      stack made changes -
      Status Resolved [ 5 ] Closed [ 6 ]
      Hide
      stack added a comment -

      Marking closed.

      Show
      stack added a comment - Marking closed.
      stack made changes -
      Fix Version/s 0.98.0 [ 12323143 ]
      stack made changes -
      Fix Version/s 0.98.0 [ 12323143 ]
      stack made changes -
      Fix Version/s 0.95.0 [ 12324094 ]
      Fix Version/s 0.96.0 [ 12320040 ]
      stack made changes -
      Status Patch Available [ 10002 ] Resolved [ 5 ]
      Hadoop Flags Reviewed [ 10343 ]
      Fix Version/s 0.96.0 [ 12320040 ]
      Resolution Fixed [ 1 ]
      Hide
      stack added a comment -

      This was committed a while back.

      Show
      stack added a comment - This was committed a while back.
      Hide
      Jesse Yates added a comment -

      The main thing I want to ensure is that we don't have a regression of HBASE-6138. I'll look at trunk soon (today?) to see if there is anything I wanna change.

      Show
      Jesse Yates added a comment - The main thing I want to ensure is that we don't have a regression of HBASE-6138 . I'll look at trunk soon (today?) to see if there is anything I wanna change.
      Hide
      stack added a comment -

      I reenabled site building and packaging up on jenkins.

      Let me know Jesse if you think we need to add other stuff from this issue to hbase-6154, the follow-on issue. Thanks.

      Show
      stack added a comment - I reenabled site building and packaging up on jenkins. Let me know Jesse if you think we need to add other stuff from this issue to hbase-6154, the follow-on issue. Thanks.
      Hide
      Hudson added a comment -

      Integrated in HBase-TRUNK #2981 (See https://builds.apache.org/job/HBase-TRUNK/2981/)
      HBASE-6145 Fix site target post modularization (Revision 1345793)

      Result = FAILURE
      stack :
      Files :

      • /hbase/trunk/src/docbkx/developer.xml
      Show
      Hudson added a comment - Integrated in HBase-TRUNK #2981 (See https://builds.apache.org/job/HBase-TRUNK/2981/ ) HBASE-6145 Fix site target post modularization (Revision 1345793) Result = FAILURE stack : Files : /hbase/trunk/src/docbkx/developer.xml
      Hide
      Hudson added a comment -

      Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #38 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/38/)
      HBASE-6145 Fix site target post modularization (Revision 1345793)
      HBASE-6145 Fix site target post modularization (Revision 1345788)

      Result = FAILURE
      stack :
      Files :

      • /hbase/trunk/src/docbkx/developer.xml

      stack :
      Files :

      • /hbase/trunk/hbase-assembly
      • /hbase/trunk/hbase-common/pom.xml
      • /hbase/trunk/hbase-server/pom.xml
      • /hbase/trunk/hbase-site
      • /hbase/trunk/pom.xml
      • /hbase/trunk/src
      • /hbase/trunk/src/assembly
      • /hbase/trunk/src/assembly/all.xml
      • /hbase/trunk/src/docbkx
      • /hbase/trunk/src/docbkx/book.xml
      • /hbase/trunk/src/docbkx/case_studies.xml
      • /hbase/trunk/src/docbkx/configuration.xml
      • /hbase/trunk/src/docbkx/customization.xsl
      • /hbase/trunk/src/docbkx/developer.xml
      • /hbase/trunk/src/docbkx/external_apis.xml
      • /hbase/trunk/src/docbkx/getting_started.xml
      • /hbase/trunk/src/docbkx/ops_mgt.xml
      • /hbase/trunk/src/docbkx/performance.xml
      • /hbase/trunk/src/docbkx/preface.xml
      • /hbase/trunk/src/docbkx/security.xml
      • /hbase/trunk/src/docbkx/shell.xml
      • /hbase/trunk/src/docbkx/troubleshooting.xml
      • /hbase/trunk/src/docbkx/upgrading.xml
      • /hbase/trunk/src/site
      • /hbase/trunk/src/site/resources
      • /hbase/trunk/src/site/resources/css
      • /hbase/trunk/src/site/resources/css/freebsd_docbook.css
      • /hbase/trunk/src/site/resources/css/site.css
      • /hbase/trunk/src/site/resources/doap_Hbase.rdf
      • /hbase/trunk/src/site/resources/images
      • /hbase/trunk/src/site/resources/images/architecture.gif
      • /hbase/trunk/src/site/resources/images/favicon.ico
      • /hbase/trunk/src/site/resources/images/hadoop-logo.jpg
      • /hbase/trunk/src/site/resources/images/hbase_logo.png
      • /hbase/trunk/src/site/resources/images/hbase_logo.svg
      • /hbase/trunk/src/site/resources/images/hfile.png
      • /hbase/trunk/src/site/resources/images/hfilev2.png
      • /hbase/trunk/src/site/resources/images/replication_overview.png
      • /hbase/trunk/src/site/site.vm
      • /hbase/trunk/src/site/site.xml
      • /hbase/trunk/src/site/xdoc
      • /hbase/trunk/src/site/xdoc/acid-semantics.xml
      • /hbase/trunk/src/site/xdoc/bulk-loads.xml
      • /hbase/trunk/src/site/xdoc/cygwin.xml
      • /hbase/trunk/src/site/xdoc/index.xml
      • /hbase/trunk/src/site/xdoc/metrics.xml
      • /hbase/trunk/src/site/xdoc/old_news.xml
      • /hbase/trunk/src/site/xdoc/pseudo-distributed.xml
      • /hbase/trunk/src/site/xdoc/replication.xml
      • /hbase/trunk/src/site/xdoc/sponsors.xml
      • /hbase/trunk/src/xslt
      • /hbase/trunk/src/xslt/configuration_to_docbook_section.xsl
      Show
      Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #38 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/38/ ) HBASE-6145 Fix site target post modularization (Revision 1345793) HBASE-6145 Fix site target post modularization (Revision 1345788) Result = FAILURE stack : Files : /hbase/trunk/src/docbkx/developer.xml stack : Files : /hbase/trunk/hbase-assembly /hbase/trunk/hbase-common/pom.xml /hbase/trunk/hbase-server/pom.xml /hbase/trunk/hbase-site /hbase/trunk/pom.xml /hbase/trunk/src /hbase/trunk/src/assembly /hbase/trunk/src/assembly/all.xml /hbase/trunk/src/docbkx /hbase/trunk/src/docbkx/book.xml /hbase/trunk/src/docbkx/case_studies.xml /hbase/trunk/src/docbkx/configuration.xml /hbase/trunk/src/docbkx/customization.xsl /hbase/trunk/src/docbkx/developer.xml /hbase/trunk/src/docbkx/external_apis.xml /hbase/trunk/src/docbkx/getting_started.xml /hbase/trunk/src/docbkx/ops_mgt.xml /hbase/trunk/src/docbkx/performance.xml /hbase/trunk/src/docbkx/preface.xml /hbase/trunk/src/docbkx/security.xml /hbase/trunk/src/docbkx/shell.xml /hbase/trunk/src/docbkx/troubleshooting.xml /hbase/trunk/src/docbkx/upgrading.xml /hbase/trunk/src/site /hbase/trunk/src/site/resources /hbase/trunk/src/site/resources/css /hbase/trunk/src/site/resources/css/freebsd_docbook.css /hbase/trunk/src/site/resources/css/site.css /hbase/trunk/src/site/resources/doap_Hbase.rdf /hbase/trunk/src/site/resources/images /hbase/trunk/src/site/resources/images/architecture.gif /hbase/trunk/src/site/resources/images/favicon.ico /hbase/trunk/src/site/resources/images/hadoop-logo.jpg /hbase/trunk/src/site/resources/images/hbase_logo.png /hbase/trunk/src/site/resources/images/hbase_logo.svg /hbase/trunk/src/site/resources/images/hfile.png /hbase/trunk/src/site/resources/images/hfilev2.png /hbase/trunk/src/site/resources/images/replication_overview.png /hbase/trunk/src/site/site.vm /hbase/trunk/src/site/site.xml /hbase/trunk/src/site/xdoc /hbase/trunk/src/site/xdoc/acid-semantics.xml /hbase/trunk/src/site/xdoc/bulk-loads.xml /hbase/trunk/src/site/xdoc/cygwin.xml /hbase/trunk/src/site/xdoc/index.xml /hbase/trunk/src/site/xdoc/metrics.xml /hbase/trunk/src/site/xdoc/old_news.xml /hbase/trunk/src/site/xdoc/pseudo-distributed.xml /hbase/trunk/src/site/xdoc/replication.xml /hbase/trunk/src/site/xdoc/sponsors.xml /hbase/trunk/src/xslt /hbase/trunk/src/xslt/configuration_to_docbook_section.xsl
      Hide
      Hudson added a comment -

      Integrated in HBase-TRUNK #2980 (See https://builds.apache.org/job/HBase-TRUNK/2980/)
      HBASE-6145 Fix site target post modularization (Revision 1345788)

      Result = FAILURE
      stack :
      Files :

      • /hbase/trunk/hbase-assembly
      • /hbase/trunk/hbase-common/pom.xml
      • /hbase/trunk/hbase-server/pom.xml
      • /hbase/trunk/hbase-site
      • /hbase/trunk/pom.xml
      • /hbase/trunk/src
      • /hbase/trunk/src/assembly
      • /hbase/trunk/src/assembly/all.xml
      • /hbase/trunk/src/docbkx
      • /hbase/trunk/src/docbkx/book.xml
      • /hbase/trunk/src/docbkx/case_studies.xml
      • /hbase/trunk/src/docbkx/configuration.xml
      • /hbase/trunk/src/docbkx/customization.xsl
      • /hbase/trunk/src/docbkx/developer.xml
      • /hbase/trunk/src/docbkx/external_apis.xml
      • /hbase/trunk/src/docbkx/getting_started.xml
      • /hbase/trunk/src/docbkx/ops_mgt.xml
      • /hbase/trunk/src/docbkx/performance.xml
      • /hbase/trunk/src/docbkx/preface.xml
      • /hbase/trunk/src/docbkx/security.xml
      • /hbase/trunk/src/docbkx/shell.xml
      • /hbase/trunk/src/docbkx/troubleshooting.xml
      • /hbase/trunk/src/docbkx/upgrading.xml
      • /hbase/trunk/src/site
      • /hbase/trunk/src/site/resources
      • /hbase/trunk/src/site/resources/css
      • /hbase/trunk/src/site/resources/css/freebsd_docbook.css
      • /hbase/trunk/src/site/resources/css/site.css
      • /hbase/trunk/src/site/resources/doap_Hbase.rdf
      • /hbase/trunk/src/site/resources/images
      • /hbase/trunk/src/site/resources/images/architecture.gif
      • /hbase/trunk/src/site/resources/images/favicon.ico
      • /hbase/trunk/src/site/resources/images/hadoop-logo.jpg
      • /hbase/trunk/src/site/resources/images/hbase_logo.png
      • /hbase/trunk/src/site/resources/images/hbase_logo.svg
      • /hbase/trunk/src/site/resources/images/hfile.png
      • /hbase/trunk/src/site/resources/images/hfilev2.png
      • /hbase/trunk/src/site/resources/images/replication_overview.png
      • /hbase/trunk/src/site/site.vm
      • /hbase/trunk/src/site/site.xml
      • /hbase/trunk/src/site/xdoc
      • /hbase/trunk/src/site/xdoc/acid-semantics.xml
      • /hbase/trunk/src/site/xdoc/bulk-loads.xml
      • /hbase/trunk/src/site/xdoc/cygwin.xml
      • /hbase/trunk/src/site/xdoc/index.xml
      • /hbase/trunk/src/site/xdoc/metrics.xml
      • /hbase/trunk/src/site/xdoc/old_news.xml
      • /hbase/trunk/src/site/xdoc/pseudo-distributed.xml
      • /hbase/trunk/src/site/xdoc/replication.xml
      • /hbase/trunk/src/site/xdoc/sponsors.xml
      • /hbase/trunk/src/xslt
      • /hbase/trunk/src/xslt/configuration_to_docbook_section.xsl
      Show
      Hudson added a comment - Integrated in HBase-TRUNK #2980 (See https://builds.apache.org/job/HBase-TRUNK/2980/ ) HBASE-6145 Fix site target post modularization (Revision 1345788) Result = FAILURE stack : Files : /hbase/trunk/hbase-assembly /hbase/trunk/hbase-common/pom.xml /hbase/trunk/hbase-server/pom.xml /hbase/trunk/hbase-site /hbase/trunk/pom.xml /hbase/trunk/src /hbase/trunk/src/assembly /hbase/trunk/src/assembly/all.xml /hbase/trunk/src/docbkx /hbase/trunk/src/docbkx/book.xml /hbase/trunk/src/docbkx/case_studies.xml /hbase/trunk/src/docbkx/configuration.xml /hbase/trunk/src/docbkx/customization.xsl /hbase/trunk/src/docbkx/developer.xml /hbase/trunk/src/docbkx/external_apis.xml /hbase/trunk/src/docbkx/getting_started.xml /hbase/trunk/src/docbkx/ops_mgt.xml /hbase/trunk/src/docbkx/performance.xml /hbase/trunk/src/docbkx/preface.xml /hbase/trunk/src/docbkx/security.xml /hbase/trunk/src/docbkx/shell.xml /hbase/trunk/src/docbkx/troubleshooting.xml /hbase/trunk/src/docbkx/upgrading.xml /hbase/trunk/src/site /hbase/trunk/src/site/resources /hbase/trunk/src/site/resources/css /hbase/trunk/src/site/resources/css/freebsd_docbook.css /hbase/trunk/src/site/resources/css/site.css /hbase/trunk/src/site/resources/doap_Hbase.rdf /hbase/trunk/src/site/resources/images /hbase/trunk/src/site/resources/images/architecture.gif /hbase/trunk/src/site/resources/images/favicon.ico /hbase/trunk/src/site/resources/images/hadoop-logo.jpg /hbase/trunk/src/site/resources/images/hbase_logo.png /hbase/trunk/src/site/resources/images/hbase_logo.svg /hbase/trunk/src/site/resources/images/hfile.png /hbase/trunk/src/site/resources/images/hfilev2.png /hbase/trunk/src/site/resources/images/replication_overview.png /hbase/trunk/src/site/site.vm /hbase/trunk/src/site/site.xml /hbase/trunk/src/site/xdoc /hbase/trunk/src/site/xdoc/acid-semantics.xml /hbase/trunk/src/site/xdoc/bulk-loads.xml /hbase/trunk/src/site/xdoc/cygwin.xml /hbase/trunk/src/site/xdoc/index.xml /hbase/trunk/src/site/xdoc/metrics.xml /hbase/trunk/src/site/xdoc/old_news.xml /hbase/trunk/src/site/xdoc/pseudo-distributed.xml /hbase/trunk/src/site/xdoc/replication.xml /hbase/trunk/src/site/xdoc/sponsors.xml /hbase/trunk/src/xslt /hbase/trunk/src/xslt/configuration_to_docbook_section.xsl
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12530710/6145v4.txt
      against trunk revision .

      -1 @author. The patch appears to contain 3 @author tags which the Hadoop community has agreed to not allow in code contributions.

      +1 tests included. The patch appears to include 16 new or modified tests.

      -1 patch. The patch command could not apply the patch.

      Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2096//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530710/6145v4.txt against trunk revision . -1 @author. The patch appears to contain 3 @author tags which the Hadoop community has agreed to not allow in code contributions. +1 tests included. The patch appears to include 16 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2096//console This message is automatically generated.
      stack made changes -
      Attachment 6145v4.txt [ 12530710 ]
      Hide
      stack added a comment -

      Here is what I actually applied to trunk.

      Show
      stack added a comment - Here is what I actually applied to trunk.
      Hide
      stack added a comment -

      I tried the test inclusion thingy:

      [INFO] --- maven-assembly-plugin:2.3:assembly (default-cli) @ hbase ---
      [INFO] Reading assembly descriptor: src/assembly/all.xml
      [WARNING] The following patterns were never triggered in this artifact inclusion filter:
      o  'org.apache.hbase:hbase-*:*:test'
      

      I also tried commenting out the binaries moduleSet for 'lib' and after all was done, my lib dir was empty so it seems like its still needed (or to put it another way, I don't currently have the energy to spend figuring out the mvn idiosyncracy that makes it so this is no longer needed – smile).

      Ok. Let me commit this last version. Then I'll go back over this issue and make more cleanup issues if we need more beyond hbase-6154.

      Thanks for the great review Jesse.

      Show
      stack added a comment - I tried the test inclusion thingy: [INFO] --- maven-assembly-plugin:2.3:assembly ( default -cli) @ hbase --- [INFO] Reading assembly descriptor: src/assembly/all.xml [WARNING] The following patterns were never triggered in this artifact inclusion filter: o 'org.apache.hbase:hbase-*:*:test' I also tried commenting out the binaries moduleSet for 'lib' and after all was done, my lib dir was empty so it seems like its still needed (or to put it another way, I don't currently have the energy to spend figuring out the mvn idiosyncracy that makes it so this is no longer needed – smile). Ok. Let me commit this last version. Then I'll go back over this issue and make more cleanup issues if we need more beyond hbase-6154. Thanks for the great review Jesse.
      Hide
      Jesse Yates added a comment -

      How did you get that Unknown macro message? I don't see it when I run local? W/ the -X flag? I see a version when avro does its stuff.

      Sorry, that was a jira issue. Apparently it doesn't like $ and { in quotes, it needs to be in the code section. See the raw output (email) for the right comment. Or just ignore the "unknown macro" comment and just read as written (would fix, but can't seem to edit jira comments).

      On avro, its deprecated. I moved it out of top-level into module where its used as a). encouraging its deprecation, and b). to get rid of some of the noise avro was generating each time mvn dipped into a module.

      Was thinking about this more last night - you make a good point. Let's keep it down in hbase-server.

      On hbase-common/pom.xml and surefire '...can/should stay in the pluginManagement section', I don't think our modules should have a pluginManagement section. It makes sense in top level or in a module IF this module had submodules but otherwise a pluginManagement doesn't make sense (as I understand it). I tried removing them from modules.

      That seems fair. Doesn't really bother me either way in the end, just wanted a good reason

      On hbase/pom.xml, assembly:assembly is not deprecated.

      It still works, but just not encouraged. It looks like its the better way to go though. +1 on that

      'In src/assembly/all.xml, this comment is no longer applicable.'

      Hmmm, that's odd. You might want to try adding a specific includes for the test classifier. Something like (untested):

      ...
      +  <moduleSets>
      +    <moduleSet>
      +      <!-- Enable access to all projects in the current multimodule build. Eclipse 
      +        says this is an error, but builds from the command line just fine. -->
      +      <useAllReactorProjects>true</useAllReactorProjects>
      +      <!-- This should work with more than 1 source module -->
      +      <!-- Now, select which projects to include in this module-set. -->
      +      <!-- Just add future modules here assuming the wildcare doesn't match -->
      +      <includes>
      +        <include>org.apache.hbase:hbase-*</include>
      +        <include>org.apache.hbase:hbase-*:*:test</include>
      +      </includes>
      

      Also, in the same file, this looks like a little cruft in the latest version:

      +      <binaries>
      +        <outputDirectory>lib</outputDirectory>
      +        <unpack>false</unpack>
      +        <dependencySets>
      +          <dependencySet/>
      +        </dependencySets>
      

      I think the dependency sets stuff can be removed.

      I don't know what NO-MVN-MAN-VER does

      Its a special tag so eclipse doesn't give a warning that you are overriding the base version. Cosmetic fix.

      I suppose. I don't trust mvn to do anything right. Therefore my tendency is to keep it dumb.

      Ok, I'll look into it in another ticket at some point.

      On docbkx, they are built into target/docbkx, and then on site build, copied under site dir. Similar to javadoc. Keeping it a little independent of site in case someone is working w/ docbkx only, and not interested in site (This is how it used work).

      Ok, +1 on all that movement. Strikes me as a bit of pain and costly, but eh.

      Let's get a couple of tickets for the cleanup/fixing so we don't forget.

      If you can address the couple of things that were missed because of jira confusion and we get good QA run, I'm good for commit.

      Show
      Jesse Yates added a comment - How did you get that Unknown macro message? I don't see it when I run local? W/ the -X flag? I see a version when avro does its stuff. Sorry, that was a jira issue. Apparently it doesn't like $ and { in quotes, it needs to be in the code section. See the raw output (email) for the right comment. Or just ignore the "unknown macro" comment and just read as written (would fix, but can't seem to edit jira comments). On avro, its deprecated. I moved it out of top-level into module where its used as a). encouraging its deprecation, and b). to get rid of some of the noise avro was generating each time mvn dipped into a module. Was thinking about this more last night - you make a good point. Let's keep it down in hbase-server. On hbase-common/pom.xml and surefire '...can/should stay in the pluginManagement section', I don't think our modules should have a pluginManagement section. It makes sense in top level or in a module IF this module had submodules but otherwise a pluginManagement doesn't make sense (as I understand it). I tried removing them from modules. That seems fair. Doesn't really bother me either way in the end, just wanted a good reason On hbase/pom.xml, assembly:assembly is not deprecated. It still works, but just not encouraged. It looks like its the better way to go though. +1 on that 'In src/assembly/all.xml, this comment is no longer applicable.' Hmmm, that's odd. You might want to try adding a specific includes for the test classifier. Something like (untested): ... + <moduleSets> + <moduleSet> + <!-- Enable access to all projects in the current multimodule build. Eclipse + says this is an error, but builds from the command line just fine. --> + <useAllReactorProjects> true </useAllReactorProjects> + <!-- This should work with more than 1 source module --> + <!-- Now, select which projects to include in this module-set. --> + <!-- Just add future modules here assuming the wildcare doesn't match --> + <includes> + <include>org.apache.hbase:hbase-*</include> + <include>org.apache.hbase:hbase-*:*:test</include> + </includes> Also, in the same file, this looks like a little cruft in the latest version: + <binaries> + <outputDirectory>lib</outputDirectory> + <unpack> false </unpack> + <dependencySets> + <dependencySet/> + </dependencySets> I think the dependency sets stuff can be removed. I don't know what NO-MVN-MAN-VER does Its a special tag so eclipse doesn't give a warning that you are overriding the base version. Cosmetic fix. I suppose. I don't trust mvn to do anything right. Therefore my tendency is to keep it dumb. Ok, I'll look into it in another ticket at some point. On docbkx, they are built into target/docbkx, and then on site build, copied under site dir. Similar to javadoc. Keeping it a little independent of site in case someone is working w/ docbkx only, and not interested in site (This is how it used work). Ok, +1 on all that movement. Strikes me as a bit of pain and costly, but eh. Let's get a couple of tickets for the cleanup/fixing so we don't forget. If you can address the couple of things that were missed because of jira confusion and we get good QA run, I'm good for commit.
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12530664/6145v4.txt
      against trunk revision .

      -1 @author. The patch appears to contain 3 @author tags which the Hadoop community has agreed to not allow in code contributions.

      +1 tests included. The patch appears to include 16 new or modified tests.

      +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      -1 core tests. The patch failed these unit tests:
      org.apache.hadoop.hbase.client.TestFromClientSide
      org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks
      org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher

      Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2089//testReport/
      Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2089//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530664/6145v4.txt against trunk revision . -1 @author. The patch appears to contain 3 @author tags which the Hadoop community has agreed to not allow in code contributions. +1 tests included. The patch appears to include 16 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSide org.apache.hadoop.hbase.io.hfile.TestForceCacheImportantBlocks org.apache.hadoop.hbase.security.access.TestZKPermissionsWatcher Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2089//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2089//console This message is automatically generated.
      Hide
      stack added a comment -

      I made HBASE-6154 to address stuff not included in this patch.

      Jesse, I won't commit, not till I get your blessing (you might have problem w/ some of my response above)

      Show
      stack added a comment - I made HBASE-6154 to address stuff not included in this patch. Jesse, I won't commit, not till I get your blessing (you might have problem w/ some of my response above)
      stack made changes -
      Attachment 6145v4.txt [ 12530664 ]
      Hide
      stack added a comment -

      Address Jesse comments.

      Show
      stack added a comment - Address Jesse comments.
      Hide
      stack added a comment -

      On avro, its deprecated. I moved it out of top-level into module where its used as a). encouraging its deprecation, and b). to get rid of some of the noise avro was generating each time mvn dipped into a module.

      How did you get that Unknown macro message? I don't see it when I run local? W/ the -X flag? I see a version when avro does its stuff.

      I moved avro back up to top-level, at least the pluginManagement section for now.

      I did pretty print on all the poms to fix indent issues: xmllint --format.

      On hbase-common/pom.xml and surefire '...can/should stay in the pluginManagement section', I don't think our modules should have a pluginManagement section. It makes sense in top level or in a module IF this module had submodules but otherwise a pluginManagement doesn't make sense (as I understand it). I tried removing them from modules.

      On site goal and the following '...but is any actual work done?'

      There is. A site dir is made w/ css and images in it. I just checked. Seems like the site dir is made anyways, in spite of these flags saying don't generate a site. I just removed them.

      On hbase/pom.xml, assembly:assembly is not deprecated. A distinction is made between assembly:single and assembly:assembly. The former is for attaching to the packaging or pre-package phase somewhere. The latter requires explicit invocation on the command-line. In my comment above at 01/Jun/12 23:24, I talk of how I looked at taking both routes and decided against the direction the sonatype manual was encouraging because a). its an ugly hack (even the manual allows so) and b). their technique attaches itself to package phase which means a user who wants to do a basic jar build has to wait on maven copying around fat dependencies and gzipping up packages all the while spewing their console though all they want to do is check their jar builds. A third reason to avoid hbase-assembly is that hbase-assembly would force an hbase-site too since hbase-assembly would want to depend on hbase-site if the tarball was to include documentation (the javadoc and jxr aggregations work fine up in parent, wasn't sure how well they'd work in a submodule and it seemed wrong doing aggregations in a submodule anyways). I think it better that we require you explicitly ask for tarball packaging by adding the assembly:assembly to your command line (I was afraid it would not work, that the dependency facility figuring which jars to include would be broken but it seems fine).

      Regards 'In src/assembly/all.xml, this comment is no longer applicable.' I think it is. At least, w/o that section, I can't get the test jar to build in (which is why you added that comment in first place I guess?).

      Also, it would be awesome to move file set matching to a more general regex, rather than tying it to the maven property (which is defined in the main pom).

      I suppose. I don't trust mvn to do anything right. Therefore my tendency is to keep it dumb.

      General nit: I prefer having the properties above the build, since the properties are used in the build section, but that's just style.

      Yeah. I kept looking for them above ... but this is not my change. This is how it was. We can change it in another issue?

      ...when building the site? Also, can't you just pass in -DskipJavadoc?

      You mean instead of -DskipJavadoc=true?

      I just tried it, and yes, seems to work. Let me change the comment.

      Again, how do you get those Unknown macro outputs? I tried w/ -X and it doesn't show.

      I don't know what NO-MVN-MAN-VER does (google'd it but no explaination after clicking ~10 links).

      Escape string is not new. Copied from what currently exists.

      On docbkx, they are built into target/docbkx, and then on site build, copied under site dir. Similar to javadoc. Keeping it a little independent of site in case someone is working w/ docbkx only, and not interested in site (This is how it used work).

      For docbkx configuration, this is how it was. I tried your suggestion of moving common config above the executions and that seems to work. Good. Fixed.

      With the aggregate goal, you can just have all the javadocs copied into the right directory in this module when you build them; at least that is what is was doing before.

      This is a change you made, that javadoc was aggregated into site/apidocs. Previous to this, pre-modularization, javadocs were made into target/apidocs and then copied into site when we ran site goal. We could go that way but seemed strange building into site if not interested in 'site': i.e. you are just making javadocs to check them out, etc.

      The xmllint --format should take care of indents and tabs.

      On fixing eclipse, can we do that in another issue. Making site and assembly work and with a patch of 1.5M, this issue is broad enough I'd say.

      Thanks for the great review. Sounds like no fundamental objection so I'll go ahead and commit. I'll open new issues to address the good stuff you raise above not addressed by this patch.

      Show
      stack added a comment - On avro, its deprecated. I moved it out of top-level into module where its used as a). encouraging its deprecation, and b). to get rid of some of the noise avro was generating each time mvn dipped into a module. How did you get that Unknown macro message? I don't see it when I run local? W/ the -X flag? I see a version when avro does its stuff. I moved avro back up to top-level, at least the pluginManagement section for now. I did pretty print on all the poms to fix indent issues: xmllint --format. On hbase-common/pom.xml and surefire '...can/should stay in the pluginManagement section', I don't think our modules should have a pluginManagement section. It makes sense in top level or in a module IF this module had submodules but otherwise a pluginManagement doesn't make sense (as I understand it). I tried removing them from modules. On site goal and the following '...but is any actual work done?' There is. A site dir is made w/ css and images in it. I just checked. Seems like the site dir is made anyways, in spite of these flags saying don't generate a site. I just removed them. On hbase/pom.xml, assembly:assembly is not deprecated. A distinction is made between assembly:single and assembly:assembly. The former is for attaching to the packaging or pre-package phase somewhere. The latter requires explicit invocation on the command-line. In my comment above at 01/Jun/12 23:24, I talk of how I looked at taking both routes and decided against the direction the sonatype manual was encouraging because a). its an ugly hack (even the manual allows so) and b). their technique attaches itself to package phase which means a user who wants to do a basic jar build has to wait on maven copying around fat dependencies and gzipping up packages all the while spewing their console though all they want to do is check their jar builds. A third reason to avoid hbase-assembly is that hbase-assembly would force an hbase-site too since hbase-assembly would want to depend on hbase-site if the tarball was to include documentation (the javadoc and jxr aggregations work fine up in parent, wasn't sure how well they'd work in a submodule and it seemed wrong doing aggregations in a submodule anyways). I think it better that we require you explicitly ask for tarball packaging by adding the assembly:assembly to your command line (I was afraid it would not work, that the dependency facility figuring which jars to include would be broken but it seems fine). Regards 'In src/assembly/all.xml, this comment is no longer applicable.' I think it is. At least, w/o that section, I can't get the test jar to build in (which is why you added that comment in first place I guess?). Also, it would be awesome to move file set matching to a more general regex, rather than tying it to the maven property (which is defined in the main pom). I suppose. I don't trust mvn to do anything right. Therefore my tendency is to keep it dumb. General nit: I prefer having the properties above the build, since the properties are used in the build section, but that's just style. Yeah. I kept looking for them above ... but this is not my change. This is how it was. We can change it in another issue? ...when building the site? Also, can't you just pass in -DskipJavadoc? You mean instead of -DskipJavadoc=true? I just tried it, and yes, seems to work. Let me change the comment. Again, how do you get those Unknown macro outputs? I tried w/ -X and it doesn't show. I don't know what NO-MVN-MAN-VER does (google'd it but no explaination after clicking ~10 links). Escape string is not new. Copied from what currently exists. On docbkx, they are built into target/docbkx, and then on site build, copied under site dir. Similar to javadoc. Keeping it a little independent of site in case someone is working w/ docbkx only, and not interested in site (This is how it used work). For docbkx configuration, this is how it was. I tried your suggestion of moving common config above the executions and that seems to work. Good. Fixed. With the aggregate goal, you can just have all the javadocs copied into the right directory in this module when you build them; at least that is what is was doing before. This is a change you made, that javadoc was aggregated into site/apidocs. Previous to this, pre-modularization, javadocs were made into target/apidocs and then copied into site when we ran site goal. We could go that way but seemed strange building into site if not interested in 'site': i.e. you are just making javadocs to check them out, etc. The xmllint --format should take care of indents and tabs. On fixing eclipse, can we do that in another issue. Making site and assembly work and with a patch of 1.5M, this issue is broad enough I'd say. Thanks for the great review. Sounds like no fundamental objection so I'll go ahead and commit. I'll open new issues to address the good stuff you raise above not addressed by this patch.
      Hide
      Jesse Yates added a comment -

      A pretty in depth analysis, hopefully not too much:
      Patch didn't apply cleanly with git, but got it go with basic patch command (mostly - looks like you might be a little behind in the docbkx?).
      I'm just going to step through per pom…

      hbase-server/pom.xml

      + <version>$

      Unknown macro: {avro.version}

      </version>

      This (and the rest of the dependency info) should be in the parent pom's dependencyManagement section. Just as general style, if another module wants to use avro, they should just declare the dependency, and not worry about making sure they have the right version, excludes, etc (I know we are dropping avro soon, but we should still do the right thing).

      Also, looks like your spacing is off for the added dependencies - 2 spaces, not tabs in xml.

      hbase-common/pom.xml

      + <plugins>
      + <plugin>
      + <artifactId>maven-surefire-plugin</artifactId>

      This can/should stay in the pluginManagement section. Surefire is part of the default maven things to run, so it will just pick up the configuration/executions from the management section - this also keeps all the surefire stuff in the same sections in all poms.

      Also, does anything actually happen if we remove:

      + <plugin>
      + <groupId>org.apache.maven.plugins</groupId>
      + <artifactId>maven-site-plugin</artifactId>

      from this pom? It may create the target/site (side effect of site being a core part of maven, so all modules can respond to it), but is any actual work done?

      Okay, onto the parent pom (/hbase/pom.xml):

      Why the removal of the hbase-assembly module? In the official docs, it actually says to use an assembly module. This is particularly poignant because the alternative is to use the assembly:assembly descriptor, which is deprecated… The docs say that you can use assembly:assembly from within the parent pom (but doesn't say what that actually means) - any way to tie the assembly:single phase to call the assembly:single/:assembly phase in the children poms?

      In src/assembly/all.xml, this comment is no longer applicable.

      <!-- This is only necessary until maven fixes the intra-project dependency bug
      in maven 3.0. Until then, we have to include the test jars for sub-projects. When
      fixed, the below dependencySet stuff is sufficient for pulling in the test jars as
      well, as long as they are added as dependencies in this project. Right now, we only
      have 1 submodule to accumulate, but we can copy/paste as necessary until maven is
      fixed. -->

      Also, it would be awesome to move file set matching to a more general regex, rather than tying it to the maven property (which is defined in the main pom).

      General nit: I prefer having the properties above the build, since the properties are used in the build section, but that's just style.

      <!-Pass -DskipJavadoc=true on command-line to skip javadoc building->

      when building the site? Also, can't you just pass in -DskipJavadoc?

      <version>$

      Unknown macro: {maven.assembly.version}

      </version>

      and

      <version>$

      Unknown macro: {maven.site.version}

      </version>

      Would be nice to add:

      <!--$NO-MVN-MAN-VER$-->
      

      to the end of the lines to remove the eclipse warning

      <plugin>
      <groupId>org.codehaus.mojo</groupId>
      <artifactId>xml-maven-plugin</artifactId>

      configuration formatting is off.

      <execution>
      <id>copy-docbkx</id>
      <goals>
      <goal>copy-resources</goal>
      </goals>
      <phase>pre-site</phase>
      <configuration>
      <outputDirectory>target/site</outputDirectory>
      <resources>
      <resource>
      <directory>$

      Unknown macro: {basedir}

      /target/docbkx</directory>
      <includes>
      <include>*/*</include>
      </includes>
      </resource>
      </resources>
      </configuration>
      </execution>
      </executions>
      <configuration>
      <escapeString>\</escapeString>
      </configuration>

      Is the escape string new here? What property are you avoiding overriding? Also, why are you moving them in the first place? You can just set the target directory to be $

      {basedir}

      /target/docbkx in the docbkx plugin:

               <groupId>com.agilejava.docbkx</groupId>
                <artifactId>docbkx-maven-plugin</artifactId>
      

      Here, you can also move the common traits to the 'top level' configuration for the plugin, then just put the differences in each execution. For example:

              <plugin>
                <groupId>com.agilejava.docbkx</groupId>
                <artifactId>docbkx-maven-plugin</artifactId>
                <version>2.0.14</version>
                <inherited>false</inherited>
                <dependencies>
      			<!--… some stuff -->
      		</dependencies>
      		<configuration>
      			<!--common configuration stuff here -->
      		</configuration>
                <executions>
                  <execution>
                    <id>multipage</id>
                    <goals>
                      <goal>generate-html</goal>
                    </goals>
                    <phase>pre-site</phase>
                    <configuration>
      			<!-- multi-page configuration stuff here -->
      			…
      

      This also seems excessive:

      <plugin>
      <artifactId>maven-resources-plugin</artifactId>
      <version>$

      Unknown macro: {maven.resources.plugin.version}

      </version>
      <inherited>false</inherited>
      <executions>
      <execution>
      <id>copy-javadocs</id>
      <goals>
      <goal>copy-resources</goal>
      </goals>
      <phase>pre-site</phase>

      With the aggregate goal, you can just have all the javadocs copied into the right directory in this module when you build them; at least that is what is was doing before.

      <artifactId>maven-assembly-plugin</artifactId>
      <version>$

      Unknown macro: {maven.assembly.version}

      </version>
      <inherited>false</inherited>
      <configuration>

      Whitespace is messed up for the configuration section (and actually the whole section - spaces, not tabs!)

      It would be sweet if we can get the eclipse natures/inclusions fixed for

      <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-eclipse-plugin</artifactId>
      <version>2.8</version>
      </plugin>

      This way the hbase top level folder wouldn't have a java nature by default. Also, for instance, we can add the assembly, docbkx, etc stuff for /hbase as src folders, and for hbase-server finally fix having to fix the classpath when you load it into eclipse (or do eclipse:eclipse). See:
      http://maven.apache.org/plugins/maven-eclipse-plugin/examples/specifying-source-path-inclusions-and-exclusions.html The latter might be a little out of scope for this ticket, but if you have the time...

      Other than that, looks good to me (didn't try building/testing yet). Good stuff stack.

      Show
      Jesse Yates added a comment - A pretty in depth analysis, hopefully not too much: Patch didn't apply cleanly with git, but got it go with basic patch command (mostly - looks like you might be a little behind in the docbkx?). I'm just going to step through per pom… hbase-server/pom.xml + <version>$ Unknown macro: {avro.version} </version> This (and the rest of the dependency info) should be in the parent pom's dependencyManagement section. Just as general style, if another module wants to use avro, they should just declare the dependency, and not worry about making sure they have the right version, excludes, etc (I know we are dropping avro soon, but we should still do the right thing). Also, looks like your spacing is off for the added dependencies - 2 spaces, not tabs in xml. hbase-common/pom.xml + <plugins> + <plugin> + <artifactId>maven-surefire-plugin</artifactId> This can/should stay in the pluginManagement section. Surefire is part of the default maven things to run, so it will just pick up the configuration/executions from the management section - this also keeps all the surefire stuff in the same sections in all poms. Also, does anything actually happen if we remove: + <plugin> + <groupId>org.apache.maven.plugins</groupId> + <artifactId>maven-site-plugin</artifactId> from this pom? It may create the target/site (side effect of site being a core part of maven, so all modules can respond to it), but is any actual work done? Okay, onto the parent pom (/hbase/pom.xml): Why the removal of the hbase-assembly module? In the official docs, it actually says to use an assembly module. This is particularly poignant because the alternative is to use the assembly:assembly descriptor, which is deprecated… The docs say that you can use assembly:assembly from within the parent pom (but doesn't say what that actually means) - any way to tie the assembly:single phase to call the assembly:single/:assembly phase in the children poms? In src/assembly/all.xml, this comment is no longer applicable. <!-- This is only necessary until maven fixes the intra-project dependency bug in maven 3.0. Until then, we have to include the test jars for sub-projects. When fixed, the below dependencySet stuff is sufficient for pulling in the test jars as well, as long as they are added as dependencies in this project. Right now, we only have 1 submodule to accumulate, but we can copy/paste as necessary until maven is fixed. --> Also, it would be awesome to move file set matching to a more general regex, rather than tying it to the maven property (which is defined in the main pom). General nit: I prefer having the properties above the build, since the properties are used in the build section, but that's just style. <!- Pass -DskipJavadoc=true on command-line to skip javadoc building -> when building the site? Also, can't you just pass in -DskipJavadoc? <version>$ Unknown macro: {maven.assembly.version} </version> and <version>$ Unknown macro: {maven.site.version} </version> Would be nice to add: <!--$NO-MVN-MAN-VER$--> to the end of the lines to remove the eclipse warning <plugin> <groupId>org.codehaus.mojo</groupId> <artifactId>xml-maven-plugin</artifactId> configuration formatting is off. <execution> <id>copy-docbkx</id> <goals> <goal>copy-resources</goal> </goals> <phase>pre-site</phase> <configuration> <outputDirectory>target/site</outputDirectory> <resources> <resource> <directory>$ Unknown macro: {basedir} /target/docbkx</directory> <includes> <include>* / *</include> </includes> </resource> </resources> </configuration> </execution> </executions> <configuration> <escapeString>\</escapeString> </configuration> Is the escape string new here? What property are you avoiding overriding? Also, why are you moving them in the first place? You can just set the target directory to be $ {basedir} /target/docbkx in the docbkx plugin: <groupId>com.agilejava.docbkx</groupId> <artifactId>docbkx-maven-plugin</artifactId> Here, you can also move the common traits to the 'top level' configuration for the plugin, then just put the differences in each execution. For example: <plugin> <groupId>com.agilejava.docbkx</groupId> <artifactId>docbkx-maven-plugin</artifactId> <version>2.0.14</version> <inherited> false </inherited> <dependencies> <!--… some stuff --> </dependencies> <configuration> <!--common configuration stuff here --> </configuration> <executions> <execution> <id>multipage</id> <goals> <goal>generate-html</goal> </goals> <phase>pre-site</phase> <configuration> <!-- multi-page configuration stuff here --> … This also seems excessive: <plugin> <artifactId>maven-resources-plugin</artifactId> <version>$ Unknown macro: {maven.resources.plugin.version} </version> <inherited>false</inherited> <executions> <execution> <id>copy-javadocs</id> <goals> <goal>copy-resources</goal> </goals> <phase>pre-site</phase> With the aggregate goal, you can just have all the javadocs copied into the right directory in this module when you build them; at least that is what is was doing before. <artifactId>maven-assembly-plugin</artifactId> <version>$ Unknown macro: {maven.assembly.version} </version> <inherited>false</inherited> <configuration> Whitespace is messed up for the configuration section (and actually the whole section - spaces, not tabs!) It would be sweet if we can get the eclipse natures/inclusions fixed for <plugin> <groupId>org.apache.maven.plugins</groupId> <artifactId>maven-eclipse-plugin</artifactId> <version>2.8</version> </plugin> This way the hbase top level folder wouldn't have a java nature by default. Also, for instance, we can add the assembly, docbkx, etc stuff for /hbase as src folders, and for hbase-server finally fix having to fix the classpath when you load it into eclipse (or do eclipse:eclipse). See: http://maven.apache.org/plugins/maven-eclipse-plugin/examples/specifying-source-path-inclusions-and-exclusions.html The latter might be a little out of scope for this ticket, but if you have the time... Other than that, looks good to me (didn't try building/testing yet). Good stuff stack.
      Hide
      stack added a comment -

      The above failure seems unrelated (and fixed since by Ted addendum I think).

      Jesse, what you think of this patch? I just messed more w/ it and I think its good enough to commit. Adds site and assembly (if I undo an assembly tgz, inside in it, I can build another assembly that runs, etc., so all source etc. and docs are present). What you think? You ok on commit?

      Show
      stack added a comment - The above failure seems unrelated (and fixed since by Ted addendum I think). Jesse, what you think of this patch? I just messed more w/ it and I think its good enough to commit. Adds site and assembly (if I undo an assembly tgz, inside in it, I can build another assembly that runs, etc., so all source etc. and docs are present). What you think? You ok on commit?
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12530620/sitev3.txt
      against trunk revision .

      -1 @author. The patch appears to contain 2 @author tags which the Hadoop community has agreed to not allow in code contributions.

      +1 tests included. The patch appears to include 8 new or modified tests.

      +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      -1 core tests. The patch failed these unit tests:
      org.apache.hadoop.hbase.master.TestHMasterRPCException

      Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2087//testReport/
      Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2087//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530620/sitev3.txt against trunk revision . -1 @author. The patch appears to contain 2 @author tags which the Hadoop community has agreed to not allow in code contributions. +1 tests included. The patch appears to include 8 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.master.TestHMasterRPCException Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2087//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2087//console This message is automatically generated.
      stack made changes -
      Attachment sitev3.txt [ 12530620 ]
      Hide
      stack added a comment -

      I went back to trying to make build work w/ hbase-assembly. It looked attractive because it almost does the right thing. The big problem w/ this route that maven wants you to take is that its bound to the package phase. That means whenever you do a mvn package, it'll take for ever was maven builds the world then copies it all over the place including jars to make you your .tar.gz. Most of the time when folks do package, they just want jars made and installed is my thinking so this would just be a fat annoyance.

      I went back to removing hbase-assembly and having assembly done by the parent. You invoke an assembly by doing assembly:assembly after doing a package and site (not assembly:single – thats something else). The attached patch is pretty much there. I need to do a bit more polishing. All src is included, its buildable and it runs. Let me do some more testing before committing.

      Show
      stack added a comment - I went back to trying to make build work w/ hbase-assembly. It looked attractive because it almost does the right thing. The big problem w/ this route that maven wants you to take is that its bound to the package phase. That means whenever you do a mvn package, it'll take for ever was maven builds the world then copies it all over the place including jars to make you your .tar.gz. Most of the time when folks do package, they just want jars made and installed is my thinking so this would just be a fat annoyance. I went back to removing hbase-assembly and having assembly done by the parent. You invoke an assembly by doing assembly:assembly after doing a package and site (not assembly:single – thats something else). The attached patch is pretty much there. I need to do a bit more polishing. All src is included, its buildable and it runs. Let me do some more testing before committing.
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12530584/site2.txt
      against trunk revision .

      -1 @author. The patch appears to contain 2 @author tags which the Hadoop community has agreed to not allow in code contributions.

      +1 tests included. The patch appears to include 8 new or modified tests.

      +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      -1 core tests. The patch failed these unit tests:
      org.apache.hadoop.hbase.client.TestFromClientSide

      Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2085//testReport/
      Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2085//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530584/site2.txt against trunk revision . -1 @author. The patch appears to contain 2 @author tags which the Hadoop community has agreed to not allow in code contributions. +1 tests included. The patch appears to include 8 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSide Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2085//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2085//console This message is automatically generated.
      stack made changes -
      Attachment site2.txt [ 12530584 ]
      Hide
      stack added a comment -

      v2 removes hbase-assembly and assembly in general – for now.

      At the moment after looking at assembly trying to make the assembly:assembly work – i.e. doing assembly up in the parent rather than down in hbase-assembly module – it seems like it could be made work (you just do assembly:assembly after doing package) but its totally a manual affair shaping the end product while meantime maven throws cryptic exceptions. It will take hours.

      Trying to read up on how others have done this is a trip through other people's misery. I'm tempted to just write a shell script to do the packaging.

      Show
      stack added a comment - v2 removes hbase-assembly and assembly in general – for now. At the moment after looking at assembly trying to make the assembly:assembly work – i.e. doing assembly up in the parent rather than down in hbase-assembly module – it seems like it could be made work (you just do assembly:assembly after doing package) but its totally a manual affair shaping the end product while meantime maven throws cryptic exceptions. It will take hours. Trying to read up on how others have done this is a trip through other people's misery. I'm tempted to just write a shell script to do the packaging.
      Hide
      stack added a comment -

      Test failures probably not related.

      I can't commit this yet until I fix assembly. Thinking on this more, the way I've done site up in parent might not jibe well doing assembly down in an assembly module (which I believe you cannot avoid).

      I love maven!

      Show
      stack added a comment - Test failures probably not related. I can't commit this yet until I fix assembly. Thinking on this more, the way I've done site up in parent might not jibe well doing assembly down in an assembly module (which I believe you cannot avoid). I love maven!
      Hide
      Hadoop QA added a comment -

      -1 overall. Here are the results of testing the latest attachment
      http://issues.apache.org/jira/secure/attachment/12530518/site.txt
      against trunk revision .

      -1 @author. The patch appears to contain 2 @author tags which the Hadoop community has agreed to not allow in code contributions.

      +1 tests included. The patch appears to include 8 new or modified tests.

      +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

      +1 javadoc. The javadoc tool did not generate any warning messages.

      +1 javac. The applied patch does not increase the total number of javac compiler warnings.

      -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail.

      +1 release audit. The applied patch does not increase the total number of release audit warnings.

      -1 core tests. The patch failed these unit tests:
      org.apache.hadoop.hbase.coprocessor.TestMasterObserver
      org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster

      Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2081//testReport/
      Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2081//console

      This message is automatically generated.

      Show
      Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12530518/site.txt against trunk revision . -1 @author. The patch appears to contain 2 @author tags which the Hadoop community has agreed to not allow in code contributions. +1 tests included. The patch appears to include 8 new or modified tests. +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to cause Findbugs (version 1.3.9) to fail. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests: org.apache.hadoop.hbase.coprocessor.TestMasterObserver org.apache.hadoop.hbase.regionserver.TestSplitTransactionOnCluster Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/2081//testReport/ Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/2081//console This message is automatically generated.
      stack made changes -
      Status Open [ 1 ] Patch Available [ 10002 ]
      Assignee stack [ stack ]
      stack made changes -
      Field Original Value New Value
      Attachment site.txt [ 12530518 ]
      Hide
      stack added a comment -

      Removes hbase-site module. Moves its content up to the parent. Assemblying site in parent as opposed to down in the hbase-site module is viable (at least as best as I could see given javadoc and jxr assemble land final product here in the parent target dir). Alternative is child module reacing around into peers and up into parent to assemble site.

      Moved avro out of parent and down into hbase-server where its used to stop it running in each module.

      Made the xref plugin do aggregation so all src shows in the xref when you click on it.

      Made the resource copy javadocs, site images, and docbook from target dir into site dir.

      Made the xml transfer work at parent level ahead of when its needed.

      Made these site building plugins not invoke child modules (they are not inherited). Made the children that don't do site skip site target.

      Added flag, javadoc.skip, so you can skip javadoc building if you just want to mess w/ poms building site w/o having to wait on javadoc each time.

      Moved the site building plugins out of pluginManagement.

      Updated site and jxr plugins to latest.

      Show
      stack added a comment - Removes hbase-site module. Moves its content up to the parent. Assemblying site in parent as opposed to down in the hbase-site module is viable (at least as best as I could see given javadoc and jxr assemble land final product here in the parent target dir). Alternative is child module reacing around into peers and up into parent to assemble site. Moved avro out of parent and down into hbase-server where its used to stop it running in each module. Made the xref plugin do aggregation so all src shows in the xref when you click on it. Made the resource copy javadocs, site images, and docbook from target dir into site dir. Made the xml transfer work at parent level ahead of when its needed. Made these site building plugins not invoke child modules (they are not inherited). Made the children that don't do site skip site target. Added flag, javadoc.skip, so you can skip javadoc building if you just want to mess w/ poms building site w/o having to wait on javadoc each time. Moved the site building plugins out of pluginManagement. Updated site and jxr plugins to latest.
      Hide
      stack added a comment -

      Site doesn't come together post maven modularization. Fix.

      Show
      stack added a comment - Site doesn't come together post maven modularization. Fix.
      stack created issue -

        People

        • Assignee:
          stack
          Reporter:
          stack
        • Votes:
          0 Vote for this issue
          Watchers:
          6 Start watching this issue

          Dates

          • Created:
            Updated:
            Resolved:

            Development