Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-6250

test-patch.sh doesn't clean up conf/*.xml files after the trunk run.

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: build
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      test-patch.sh doesn't clean up conf/*.xml files after trunk run. This is a problem as after applying patch , the *.xml files are not updated by the template files.

      1. HADOOP-6250.patch
        0.6 kB
        rahul k singh
      2. HADOOP-6250-2.patch
        0.7 kB
        rahul k singh
      3. HADOOP-6250-3.patch
        0.7 kB
        Hemanth Yamijala

        Activity

        Hide
        rksingh rahul k singh added a comment -

        test-patch.sh does the following in checkout() method.
        "
        $SVN revert -R .
        rm -rf `$SVN status --no-ignore`
        $SVN update
        "

        This makes sure that any .xml files in conf directory is deleted and updated from .template .

        test-patch.sh first runs the tests on trunk . This results in generation of these xml files again in conf directory , hence when patch is applied the new template file changes
        are not copied to existing xml file , this results in test case failures .

        Show
        rksingh rahul k singh added a comment - test-patch.sh does the following in checkout() method. " $SVN revert -R . rm -rf `$SVN status --no-ignore` $SVN update " This makes sure that any .xml files in conf directory is deleted and updated from .template . test-patch.sh first runs the tests on trunk . This results in generation of these xml files again in conf directory , hence when patch is applied the new template file changes are not copied to existing xml file , this results in test case failures .
        Hide
        gkesavan Giridharan Kesavan added a comment -

        I think this had to be addressed in the build.xml, perhaps we can make the ant clean target to remove the xml files.

        Show
        gkesavan Giridharan Kesavan added a comment - I think this had to be addressed in the build.xml, perhaps we can make the ant clean target to remove the xml files.
        Hide
        yhemanth Hemanth Yamijala added a comment -

        I think this had to be addressed in the build.xml, perhaps we can make the ant clean target to remove the xml files.

        I am not sure if this is the right solution. We don't overwrite xml files in the conf folder because they may have user configured values. It seems to me this is only a problem in the build setup. Hence, Rahul's solution of removing the ignored files after applying the patch in test-patch.sh seems fine to me.

        Show
        yhemanth Hemanth Yamijala added a comment - I think this had to be addressed in the build.xml, perhaps we can make the ant clean target to remove the xml files. I am not sure if this is the right solution. We don't overwrite xml files in the conf folder because they may have user configured values. It seems to me this is only a problem in the build setup. Hence, Rahul's solution of removing the ignored files after applying the patch in test-patch.sh seems fine to me.
        Hide
        rksingh rahul k singh added a comment -

        attaching the patch which cleans up the conf/*.xml for template files

        Show
        rksingh rahul k singh added a comment - attaching the patch which cleans up the conf/*.xml for template files
        Hide
        szetszwo Tsz Wo Nicholas Sze added a comment -

        What happens if there are user xml files in conf? Would these files be deleted by test-patch.sh?

        Show
        szetszwo Tsz Wo Nicholas Sze added a comment - What happens if there are user xml files in conf? Would these files be deleted by test-patch.sh?
        Hide
        rksingh rahul k singh added a comment -

        Patch only delete files , if they have corresponding template files.

        Show
        rksingh rahul k singh added a comment - Patch only delete files , if they have corresponding template files.
        Hide
        yhemanth Hemanth Yamijala added a comment -

        I think this patch is fine, except, I think this should be done only if its being run on HUDSON. That will keep parity with what is done in the checkout method and thus not cause side effects to developers using test-patch. Rahul, since we can't run this patch itself through hudson, can you please make sure that test-patch is functional when used with projects like map-reduce and then I can commit it to unblock MAPREDUCE-861 ?

        Show
        yhemanth Hemanth Yamijala added a comment - I think this patch is fine, except, I think this should be done only if its being run on HUDSON. That will keep parity with what is done in the checkout method and thus not cause side effects to developers using test-patch. Rahul, since we can't run this patch itself through hudson, can you please make sure that test-patch is functional when used with projects like map-reduce and then I can commit it to unblock MAPREDUCE-861 ?
        Hide
        rksingh rahul k singh added a comment -

        Attaching the new patch with hemanth's comments.

        Show
        rksingh rahul k singh added a comment - Attaching the new patch with hemanth's comments.
        Hide
        yhemanth Hemanth Yamijala added a comment -

        Attaching a new patch that just formats the new method as per convention. I ran test-patch locally with a Map/Reduce patch and verified it still works fine. Will commit the patch based on this.

        Show
        yhemanth Hemanth Yamijala added a comment - Attaching a new patch that just formats the new method as per convention. I ran test-patch locally with a Map/Reduce patch and verified it still works fine. Will commit the patch based on this.
        Hide
        yhemanth Hemanth Yamijala added a comment -

        I committed this to trunk. Thanks, Rahul !

        Show
        yhemanth Hemanth Yamijala added a comment - I committed this to trunk. Thanks, Rahul !
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #32 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/32/)
        . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #32 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk-Commit/32/ ) . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #35 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/35/)
        . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #35 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk-Commit/35/ ) . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #83 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/83/)
        . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #83 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk/83/ ) . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #39 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/39/)
        . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #39 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/39/ ) . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.
        Hide
        hudson Hudson added a comment -

        Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #29 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/29/)
        . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.

        Show
        hudson Hudson added a comment - Integrated in Hdfs-Patch-h5.grid.sp2.yahoo.net #29 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h5.grid.sp2.yahoo.net/29/ ) . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Common-trunk #97 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/97/)
        . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Common-trunk #97 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Common-trunk/97/ ) . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.
        Hide
        hudson Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #85 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/85/)
        . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.

        Show
        hudson Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #85 (See http://hudson.zones.apache.org/hudson/job/Hadoop-Hdfs-trunk/85/ ) . Modify test-patch to delete copied XML files before running patch build. Contributed by Rahul Kumar Singh.
        Hide
        hudson Hudson added a comment -

        Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #11 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/11/)

        Show
        hudson Hudson added a comment - Integrated in Hdfs-Patch-h2.grid.sp2.yahoo.net #11 (See http://hudson.zones.apache.org/hudson/job/Hdfs-Patch-h2.grid.sp2.yahoo.net/11/ )

          People

          • Assignee:
            rksingh rahul k singh
            Reporter:
            rksingh rahul k singh
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development