Uploaded image for project: 'OFBiz'
  1. OFBiz
  2. OFBIZ-9262

Error message when svn updating due to pullAllPluginsSource

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Trunk
    • Fix Version/s: Upcoming Release
    • Component/s: framework
    • Labels:
      None

      Description

      When try to update a ofbiz-framework working copy after having used pullAllPluginsSource, you get an error message "Skipped obstructing working copy".

      This is because we have already a plugins folder in the ofbiz-framework/trunk branch and when we use pullAllPluginsSource we replace it by a new one (plugins folder) and the main .svn gets confused (in root)

      In other words, because of the plugins/README.txt file when you create a working copy from the ofbiz-framework/trunk branch you generate a .svn in root folder where there is a "knowledge" of this file and the plugins directory.

      So, we can't delete the whole plugins directory and replace it by another different plugins directory with the plugins sub-folders inside.

      Morevoer you can't make a chekout (which pullAllPluginsSource currently does) in a non empty folder.

      And If you temporarily move the plugins/README.txt file to get an empty directory you get this error message

      > svn-checkout failed for https://svn.apache.org/repos/asf/ofbiz/ofbiz-plugins/trunk
      svn: E155000: 'C:\projectsASF\ofbiz-framework\plugins' is already a working copy for a different URL; perform update to complete it.

      1. OFBIZ-9262.patch
        1 kB
        Taher Alkhateeb

        Activity

        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        I completed the documentation at r1787307, feel free to amend...

        Show
        jacques.le.roux Jacques Le Roux added a comment - I completed the documentation at r1787307, feel free to amend...
        Hide
        taher Taher Alkhateeb added a comment -

        Okay, after digging through the logs I figured out what was going on! It simply called the cleanupBeforePulling task because of the regex matching for cleanAll. I simply renamed cleanupBeforePulling to deleteBeforePulling and now the issue is gone. The fix is in commit 1787183.

        Thank you Wai, you discovered a well hidden bug.

        Show
        taher Taher Alkhateeb added a comment - Okay, after digging through the logs I figured out what was going on! It simply called the cleanupBeforePulling task because of the regex matching for cleanAll. I simply renamed cleanupBeforePulling to deleteBeforePulling and now the issue is gone. The fix is in commit 1787183. Thank you Wai, you discovered a well hidden bug.
        Hide
        wt Wai added a comment -

        I'm currently using rev1787172 of the ofbiz trunk. After running "$ gradlew cleanAll", the plugins/ directory is removed.

        Show
        wt Wai added a comment - I'm currently using rev1787172 of the ofbiz trunk. After running "$ gradlew cleanAll", the plugins/ directory is removed.
        Hide
        taher Taher Alkhateeb added a comment -

        How did you come up with the conclusion that the cleanAll task deletes plugins? From the code in front of me this is not true

        Show
        taher Taher Alkhateeb added a comment - How did you come up with the conclusion that the cleanAll task deletes plugins? From the code in front of me this is not true
        Hide
        wt Wai added a comment -

        I've discovered an issue. This related to OFBIZ-9244 where it was agreed upon where all plugin components would be situated in plugins/ directory. This includes any proprietary components currently in development. I've found that executing "$ gradlew cleanAll" would remove the plugins/ directory and hence all current development sources. This means that a developer who is developing his component against the trunk sources would have all his component sources deleted when a "cleanAll" is executed. This is a serious issue.

        Show
        wt Wai added a comment - I've discovered an issue. This related to OFBIZ-9244 where it was agreed upon where all plugin components would be situated in plugins/ directory. This includes any proprietary components currently in development. I've found that executing "$ gradlew cleanAll" would remove the plugins/ directory and hence all current development sources. This means that a developer who is developing his component against the trunk sources would have all his component sources deleted when a "cleanAll" is executed. This is a serious issue.
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        plugins dir removed at r1787143
        Taher's patch committed at r1787144

        I'll continue with the documentation....

        Show
        jacques.le.roux Jacques Le Roux added a comment - plugins dir removed at r1787143 Taher's patch committed at r1787144 I'll continue with the documentation....
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        Thanks Deepak,

        That's what I thought, better not add ignore. It concerns only committers, if a committer want to add a new component, etc. it will be easier.

        I'll proceed now and will begin by documenting the change in the main README.md and source repositories page. Any ideas about documenting this better?

        Show
        jacques.le.roux Jacques Le Roux added a comment - Thanks Deepak, That's what I thought, better not add ignore. It concerns only committers, if a committer want to add a new component, etc. it will be easier. I'll proceed now and will begin by documenting the change in the main README.md and source repositories page. Any ideas about documenting this better?
        Hide
        deepak.dixit Deepak Dixit added a comment -

        I think we can proceed with this change, without ignore,

        In case of ignore svn will not include plugins directory, like if we have some custom changes or custom component in plugins directory then svn will ignore them while committing code in ofbiz-framework. like build or runtime/uploads etc.

        Show
        deepak.dixit Deepak Dixit added a comment - I think we can proceed with this change, without ignore, In case of ignore svn will not include plugins directory, like if we have some custom changes or custom component in plugins directory then svn will ignore them while committing code in ofbiz-framework. like build or runtime/uploads etc.
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        OK, if we remove it will work of course. But then I'm stil waiting for my unanswered question: why ignore?

        Show
        jacques.le.roux Jacques Le Roux added a comment - OK, if we remove it will work of course. But then I'm stil waiting for my unanswered question: why ignore?
        Hide
        taher Taher Alkhateeb added a comment -

        My patch assumes both removal and svn ignore

        Show
        taher Taher Alkhateeb added a comment - My patch assumes both removal and svn ignore
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        I still believe we will not get out of this pitfall w/o dropping the plugins dir. It's not so dramatic, there is nothing there but a readme! Now we need to document it, et voilà.

        Show
        jacques.le.roux Jacques Le Roux added a comment - I still believe we will not get out of this pitfall w/o dropping the plugins dir. It's not so dramatic, there is nothing there but a readme! Now we need to document it, et voilà.
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        Nope, even after committing "svn:ignore plugins"
        I get this error (I got the same before committing)

        Execution failed for task ':pullPluginsFromSvn'.
        > svn-checkout failed for https://svn.apache.org/repos/asf/ofbiz/ofbiz-plugins/trunk
        svn: E155000: Revision 1 787 108 doesnt match existing revision {1} in '{2}'
        
        Show
        jacques.le.roux Jacques Le Roux added a comment - Nope, even after committing "svn:ignore plugins" I get this error (I got the same before committing) Execution failed for task ':pullPluginsFromSvn'. > svn-checkout failed for https: //svn.apache.org/repos/asf/ofbiz/ofbiz-plugins/trunk svn: E155000: Revision 1 787 108 doesnt match existing revision {1} in '{2}'
        Hide
        taher Taher Alkhateeb added a comment -

        Okay, so I think if we set subversion to ignore the plugins directory then the attached patch can take care of the rest. Please test and advise

        Show
        taher Taher Alkhateeb added a comment - Okay, so I think if we set subversion to ignore the plugins directory then the attached patch can take care of the rest. Please test and advise
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        Pierre,

        For create component, we just need to check if the plugins dir exists and if not to create it.

        Show
        jacques.le.roux Jacques Le Roux added a comment - Pierre, For create component, we just need to check if the plugins dir exists and if not to create it.
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        Deepak, Taher,

        A commit of the plugins dir removing will work. Then an up to date working copy based on ofbiz-framework branch will know nothing about the plugins dir so will ignore what will happen there. We need to document in the README.md for users and actually maybe a bit more in the source repositories page. Note that this was my proposition from start But maybe I miss something: what adds the svn:ignore?

        I agree about checking out directly in plugins. And of course pullAllPluginsSource is only intented for committers, to be documented also (if not yet).

        Show
        jacques.le.roux Jacques Le Roux added a comment - Deepak, Taher, A commit of the plugins dir removing will work. Then an up to date working copy based on ofbiz-framework branch will know nothing about the plugins dir so will ignore what will happen there. We need to document in the README.md for users and actually maybe a bit more in the source repositories page. Note that this was my proposition from start But maybe I miss something: what adds the svn:ignore? I agree about checking out directly in plugins. And of course pullAllPluginsSource is only intented for committers, to be documented also (if not yet).
        Hide
        taher Taher Alkhateeb added a comment -

        I have a simple suggested solution based on Deepak's feedback:

        • let subversion ignore the plugins directory
        • on every run of the buildscript we check if the plugins folder exists. If it does not, then create it.

        That's it, and it would solve the subversion issue.

        WDYT?

        Show
        taher Taher Alkhateeb added a comment - I have a simple suggested solution based on Deepak's feedback: let subversion ignore the plugins directory on every run of the buildscript we check if the plugins folder exists. If it does not, then create it. That's it, and it would solve the subversion issue. WDYT?
        Hide
        deepak.dixit Deepak Dixit added a comment -

        I agree, we can remove the plugins folder and can ignore it as well.
        I had a discussion with Taher regarding same, we agreed on following:

        • Add svn:ignore on plugins (remove plugins directory )
        • Fix build process and related task, to check plugins directory and create if missing, like OFBiz do it for log directory
        • Do checkout of plugins directly in plugins directly, create plugins folder if not exists. No need to do checkout in temp and move it to another location
        • pullAllPluginSource task is for convenient to do checkout of all the plugins in one shot, its not recommended for custom development, in that case use pullPluginSource task to checkout specific plugin.
        Show
        deepak.dixit Deepak Dixit added a comment - I agree, we can remove the plugins folder and can ignore it as well. I had a discussion with Taher regarding same, we agreed on following: Add svn:ignore on plugins (remove plugins directory ) Fix build process and related task, to check plugins directory and create if missing, like OFBiz do it for log directory Do checkout of plugins directly in plugins directly, create plugins folder if not exists. No need to do checkout in temp and move it to another location pullAllPluginSource task is for convenient to do checkout of all the plugins in one shot, its not recommended for custom development, in that case use pullPluginSource task to checkout specific plugin.
        Hide
        pfm.smits Pierre Smits added a comment -

        That means the convenient create component function will fail.

        Show
        pfm.smits Pierre Smits added a comment - That means the convenient create component function will fail.
        Hide
        jacques.le.roux Jacques Le Roux added a comment - - edited

        Another (IMO) even better solution: remove the plugins folder and document it in the main README.MD file .-

        Show
        jacques.le.roux Jacques Le Roux added a comment - - edited Another (IMO) even better solution: remove the plugins folder and document it in the main README.MD file .-
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        Another solution is to have as much as WC than plugins, something I was worrying about initially...

        Show
        jacques.le.roux Jacques Le Roux added a comment - Another solution is to have as much as WC than plugins, something I was worrying about initially...
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        Ok I tried this

        Index: build.gradle
        ===================================================================
        --- build.gradle	(revision 1787060)
        +++ build.gradle	(working copy)
        @@ -17,6 +17,7 @@
          * under the License.
          */
         import at.bxm.gradleplugins.svntools.tasks.SvnCheckout
        +import at.bxm.gradleplugins.svntools.tasks.SvnUpdate
         import org.apache.tools.ant.filters.ReplaceTokens
        
         /* ========================================================
        @@ -749,10 +750,16 @@
        
             task pullPluginsFromSvn(type: SvnCheckout) {
                 svnUrl = "https://svn.apache.org/repos/asf/ofbiz/ofbiz-plugins/trunk"
        -        workspaceDir = "${rootDir}/temp"
        +        delete "${rootDir}/pluginsTemp"
        +        workspaceDir = "${rootDir}/pluginsTemp"
                 doLast{
                     delete "${rootDir}/plugins"
        -            ant.move(file: "${rootDir}/temp", toFile: "${rootDir}/plugins")
        +            ant.move(file: "${rootDir}/pluginsTemp", toFile: "${rootDir}/plugins")
        +            task pullPluginReadmeTxt(type: SvnUpdate) {
        +                svnUrl = "https://svn.apache.org/repos/asf/ofbiz/ofbiz-plugins/trunk/plugins"
        +                workspaceDir = "${rootDir}/plugins"
        +            }
        +            delete "${rootDir}/pluginsTemp"
                 }
             }
             dependsOn pullPluginsFromSvn
        

        But despite

        C:\projectsASF\ofbiz-framework>gradlew pullAllPluginsSource
        :pullPluginsFromSvn
        :pullAllPluginsSource
        :installAllPlugins UP-TO-DATE
        BUILD SUCCESSFUL
        Total time: 6.115 secs
        BUILD SUCCESSFUL
        Total time: 1 mins 28.741 secs
        C:\projectsASF\ofbiz-framework>
        

        I'm back to the same situation (ie same error msg here when udpating the whole).
        Which makes sens when you think about it. The task pullPluginReadmeTxt is actually updating the new plugins directory which works. And updating the whole from Gradle would result with the same error msg reported here. Not sure if it's possible to handle w/o an externals...

        Ah last note, I used pluginsTemp instead of temp because a custom project might use a temp directory and would be surprise to find its files in the plugins directory and possibly deleted later. I also delete pluginsTemp I suppose the name is sufficiently different what we could expect.

        Show
        jacques.le.roux Jacques Le Roux added a comment - Ok I tried this Index: build.gradle =================================================================== --- build.gradle (revision 1787060) +++ build.gradle (working copy) @@ -17,6 +17,7 @@ * under the License. */ import at.bxm.gradleplugins.svntools.tasks.SvnCheckout + import at.bxm.gradleplugins.svntools.tasks.SvnUpdate import org.apache.tools.ant.filters.ReplaceTokens /* ======================================================== @@ -749,10 +750,16 @@ task pullPluginsFromSvn(type: SvnCheckout) { svnUrl = "https: //svn.apache.org/repos/asf/ofbiz/ofbiz-plugins/trunk" - workspaceDir = "${rootDir}/temp" + delete "${rootDir}/pluginsTemp" + workspaceDir = "${rootDir}/pluginsTemp" doLast{ delete "${rootDir}/plugins" - ant.move(file: "${rootDir}/temp" , toFile: "${rootDir}/plugins" ) + ant.move(file: "${rootDir}/pluginsTemp" , toFile: "${rootDir}/plugins" ) + task pullPluginReadmeTxt(type: SvnUpdate) { + svnUrl = "https: //svn.apache.org/repos/asf/ofbiz/ofbiz-plugins/trunk/plugins" + workspaceDir = "${rootDir}/plugins" + } + delete "${rootDir}/pluginsTemp" } } dependsOn pullPluginsFromSvn But despite C:\projectsASF\ofbiz-framework>gradlew pullAllPluginsSource :pullPluginsFromSvn :pullAllPluginsSource :installAllPlugins UP-TO-DATE BUILD SUCCESSFUL Total time: 6.115 secs BUILD SUCCESSFUL Total time: 1 mins 28.741 secs C:\projectsASF\ofbiz-framework> I'm back to the same situation (ie same error msg here when udpating the whole). Which makes sens when you think about it. The task pullPluginReadmeTxt is actually updating the new plugins directory which works. And updating the whole from Gradle would result with the same error msg reported here. Not sure if it's possible to handle w/o an externals... Ah last note, I used pluginsTemp instead of temp because a custom project might use a temp directory and would be surprise to find its files in the plugins directory and possibly deleted later. I also delete pluginsTemp I suppose the name is sufficiently different what we could expect.
        Hide
        jacques.le.roux Jacques Le Roux added a comment -

        I continue with another idea..

        Show
        jacques.le.roux Jacques Le Roux added a comment - I continue with another idea..

          People

          • Assignee:
            jacques.le.roux Jacques Le Roux
            Reporter:
            jacques.le.roux Jacques Le Roux
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development