Groovy
  1. Groovy
  2. GROOVY-4014

Provide more Feedback while downloading Grapes

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.7.0
    • Fix Version/s: 1.7.6, 1.8-beta-3
    • Component/s: Grape
    • Labels:
      None

      Description

      Grape is a great System... but there is one point that is somehow annoying.
      When you download Grapes with huge dependencies it will take some time.
      From the user perspective it seems like the application is crashed.

      Is it possible to inject a Closure into Grape that gives information about the downloading state?

      Like:

      @GrabFeedback({percentage -> println "$

      {percentage}

      % downloaded"})

      This Problem also affects the Eclipse plugin. When you clear the grape cache and you restart the Eclipse, it may take minutes until eclipse becomes responsive.

        Issue Links

          Activity

          Sebastian Rühl created issue -
          Hide
          Sebastian Rühl added a comment -

          Just an Update...
          I looked tru the Ivy Src:

          there is a TransferListener (org.apache.ivy.plugins.repository.TransferListener) which can be appended to the EventManager in Ivy.

          So it could be possible to implement a Attribute in GrapeConfig like

          @GrapeConfig(downloadFeedback:{println evt.getElapsedTime(TransferEvent.TRANSFER_STARTED, TransferEvent.TRANSFER_COMPLETED)})
          

          (see Ivy.java line 304 from latest source for reference)
          In this case we would print the elapsed time. (see TransferEvent javaDoc)
          here a snippet

          Ivy.java
          ...
          eventManager.addTransferListener(new TransferListener() {
                          public void transferProgress(TransferEvent evt) {
                              ResolveData resolve;
                              switch (evt.getEventType()) {
                                  case TransferEvent.TRANSFER_PROGRESS:
                                      resolve = IvyContext.getContext().getResolveData();
                                      if (resolve == null || !LogOptions.LOG_QUIET.equals(
                                              resolve.getOptions().getLog())) {
                                          Message.progress();
                                      }
                                      break;
                                  case TransferEvent.TRANSFER_COMPLETED:
                                      resolve = IvyContext.getContext().getResolveData();
                                      if (resolve == null || !LogOptions.LOG_QUIET.equals(
                                              resolve.getOptions().getLog())) {
                                          Message.endProgress(" (" + (evt.getTotalLength() / KILO) + "kB)");
                                      }
                                      break;
                                  default:
                                      break;
                              }
                          }
                      });
          ...
          

          This listener could then appended via

          ivyInstance.eventManager.addTransferListener(callBackClosure as TransferListener)
          

          Haven't tried it yet but it seems as a possible Solution.

          Show
          Sebastian Rühl added a comment - Just an Update... I looked tru the Ivy Src: there is a TransferListener (org.apache.ivy.plugins.repository.TransferListener) which can be appended to the EventManager in Ivy. So it could be possible to implement a Attribute in GrapeConfig like @GrapeConfig(downloadFeedback:{println evt.getElapsedTime(TransferEvent.TRANSFER_STARTED, TransferEvent.TRANSFER_COMPLETED)}) (see Ivy.java line 304 from latest source for reference) In this case we would print the elapsed time. (see TransferEvent javaDoc) here a snippet Ivy.java ... eventManager.addTransferListener( new TransferListener() { public void transferProgress(TransferEvent evt) { ResolveData resolve; switch (evt.getEventType()) { case TransferEvent.TRANSFER_PROGRESS: resolve = IvyContext.getContext().getResolveData(); if (resolve == null || !LogOptions.LOG_QUIET.equals( resolve.getOptions().getLog())) { Message.progress(); } break ; case TransferEvent.TRANSFER_COMPLETED: resolve = IvyContext.getContext().getResolveData(); if (resolve == null || !LogOptions.LOG_QUIET.equals( resolve.getOptions().getLog())) { Message.endProgress( " (" + (evt.getTotalLength() / KILO) + "kB)" ); } break ; default : break ; } } }); ... This listener could then appended via ivyInstance.eventManager.addTransferListener(callBackClosure as TransferListener) Haven't tried it yet but it seems as a possible Solution.
          Hide
          Sebastian Rühl added a comment -

          Ok just a small mistake on my side.
          Annotations support only primitive types.
          So one solution is to give the Closure as as String

          @GrapeConfig(downloadFeedback:"{println evt.getElapsedTime(TransferEvent.TRANSFER_STARTED, TransferEvent.TRANSFER_COMPLETED)}")
          

          or just set a Flag:

          @GrapeConfig(downloadFeedback:true)
          

          For this scenario we must inject the Closure into the ClassLoader.

          new GroovyClassLoader().setDownloadFeedback({
            println evt.getElapsedTime(TransferEvent.TRANSFER_STARTED, TransferEvent.TRANSFER_COMPLETED)
          })
          

          I don't know what is the better way for this...

          Show
          Sebastian Rühl added a comment - Ok just a small mistake on my side. Annotations support only primitive types. So one solution is to give the Closure as as String @GrapeConfig(downloadFeedback: "{println evt.getElapsedTime(TransferEvent.TRANSFER_STARTED, TransferEvent.TRANSFER_COMPLETED)}" ) or just set a Flag: @GrapeConfig(downloadFeedback: true ) For this scenario we must inject the Closure into the ClassLoader. new GroovyClassLoader().setDownloadFeedback({ println evt.getElapsedTime(TransferEvent.TRANSFER_STARTED, TransferEvent.TRANSFER_COMPLETED) }) I don't know what is the better way for this...
          Hide
          Paul King added a comment -

          Had a quick attempt to find a simple solution as per attached patch.

          It has numerous limitations:

          • Currently prints to System.err.println (no more formal logging)
          • Not tested in Eclipse environment
          • No feedback at start; instead feedback after each @Grab which causes some download (so will be good if you have many grabs but otherwise not much different)
          • Report is too coarse grained (feedback happens at end of each @Grab not at end of each sub-jar in the transitive dependency list and all dependencies are shown even if some are already downloaded; i.e. no indication of which jars were actually downloaded is displayed)
          • Report is cumulative (dependencies of earlier @Grabs will appear in dependency list for a later @Grab but give 0 towards bytes downloaded)

          But even so, I have included it here in case it provides a useful starting point for a more sophisticated solution to this problem.

          Show
          Paul King added a comment - Had a quick attempt to find a simple solution as per attached patch. It has numerous limitations: Currently prints to System.err.println (no more formal logging) Not tested in Eclipse environment No feedback at start; instead feedback after each @Grab which causes some download (so will be good if you have many grabs but otherwise not much different) Report is too coarse grained (feedback happens at end of each @Grab not at end of each sub-jar in the transitive dependency list and all dependencies are shown even if some are already downloaded; i.e. no indication of which jars were actually downloaded is displayed) Report is cumulative (dependencies of earlier @Grabs will appear in dependency list for a later @Grab but give 0 towards bytes downloaded) But even so, I have included it here in case it provides a useful starting point for a more sophisticated solution to this problem.
          Paul King made changes -
          Field Original Value New Value
          Attachment GrabFeedback.patch [ 47802 ]
          Paul King made changes -
          Attachment GrabFeedback.patch [ 47803 ]
          Paul King made changes -
          Attachment GrabFeedback.patch [ 47802 ]
          Hide
          Alexander Kleymenov added a comment -

          While this feature is not implemented, is it somehow possible to show action log? Script idleness sometimes missleading.

          Show
          Alexander Kleymenov added a comment - While this feature is not implemented, is it somehow possible to show action log? Script idleness sometimes missleading.
          Paul King made changes -
          Link This issue relates to GROOVY-4424 [ GROOVY-4424 ]
          Hide
          Paul King added a comment -

          Revised patch. Now removes some of the limitations mentioned earlier at the expense of keeping some String sets in memory.

          Show
          Paul King added a comment - Revised patch. Now removes some of the limitations mentioned earlier at the expense of keeping some String sets in memory.
          Paul King made changes -
          Attachment grapeivy_groovy4014.patch [ 51261 ]
          Hide
          Guillaume Delcroix added a comment -

          I tried the patch and saw the following:

          glaforge-2:18 glaforge$ groovy -Dgrape.report.downloads=true -e "@Grab(group='org.springframework', module='spring-context', version='3.0.4.RELEASE') import org.springframework.context.ApplicationContext; println ApplicationContext"
          Resolving dependency: org.springframework#spring-context;3.0.4.RELEASE {default=[default]}
          Preparing to download artifact org.springframework#spring-context;3.0.4.RELEASE!spring-context.jar
          Preparing to download artifact org.springframework#spring-aop;3.0.4.RELEASE!spring-aop.jar
          Preparing to download artifact org.springframework#spring-beans;3.0.4.RELEASE!spring-beans.jar
          Preparing to download artifact org.springframework#spring-core;3.0.4.RELEASE!spring-core.jar
          Preparing to download artifact org.springframework#spring-expression;3.0.4.RELEASE!spring-expression.jar
          Preparing to download artifact org.springframework#spring-asm;3.0.4.RELEASE!spring-asm.jar
          Preparing to download artifact aopalliance#aopalliance;1.0!aopalliance.jar
          Preparing to download artifact commons-logging#commons-logging;1.1.1!commons-logging.jar
          interface org.springframework.context.ApplicationContext
          

          Nice to see what Grape's doing!
          Now if we could register listners to notify the Groovy shell and console, that would be cool. I hope we'll find a way.

          Show
          Guillaume Delcroix added a comment - I tried the patch and saw the following: glaforge-2:18 glaforge$ groovy -Dgrape.report.downloads= true -e "@Grab(group='org.springframework', module='spring-context', version='3.0.4.RELEASE') import org.springframework.context.ApplicationContext; println ApplicationContext" Resolving dependency: org.springframework#spring-context;3.0.4.RELEASE { default =[ default ]} Preparing to download artifact org.springframework#spring-context;3.0.4.RELEASE!spring-context.jar Preparing to download artifact org.springframework#spring-aop;3.0.4.RELEASE!spring-aop.jar Preparing to download artifact org.springframework#spring-beans;3.0.4.RELEASE!spring-beans.jar Preparing to download artifact org.springframework#spring-core;3.0.4.RELEASE!spring-core.jar Preparing to download artifact org.springframework#spring-expression;3.0.4.RELEASE!spring-expression.jar Preparing to download artifact org.springframework#spring-asm;3.0.4.RELEASE!spring-asm.jar Preparing to download artifact aopalliance#aopalliance;1.0!aopalliance.jar Preparing to download artifact commons-logging#commons-logging;1.1.1!commons-logging.jar interface org.springframework.context.ApplicationContext Nice to see what Grape's doing! Now if we could register listners to notify the Groovy shell and console, that would be cool. I hope we'll find a way.
          Hide
          Guillaume Delcroix added a comment -

          By the way, the system property should probably be named with a groovy prefix, like the other existing ones:
          groovy.grape.enable and groovy.grape.autoDownload.

          Show
          Guillaume Delcroix added a comment - By the way, the system property should probably be named with a groovy prefix, like the other existing ones: groovy.grape.enable and groovy.grape.autoDownload.
          Hide
          Guillaume Delcroix added a comment -

          The patch touches the Groovy console so as to provide status messages when dependencies are resolved and being downloaded.

          Show
          Guillaume Delcroix added a comment - The patch touches the Groovy console so as to provide status messages when dependencies are resolved and being downloaded.
          Guillaume Delcroix made changes -
          Attachment Add_status_messages_to_Groovy_console_providing_feedback_on_what_grape_is_downloading.patch [ 51263 ]
          Hide
          Paul King added a comment -

          Some additional logging added. Use -Dgroovy.grapes.report.downloads=true to enable. Set via JAVA_OPTS if using the windows batch file.

          Show
          Paul King added a comment - Some additional logging added. Use -Dgroovy.grapes.report.downloads=true to enable. Set via JAVA_OPTS if using the windows batch file.
          Paul King made changes -
          Resolution Fixed [ 1 ]
          Assignee Paul King [ paulk ]
          Fix Version/s 1.8-beta-3 [ 16750 ]
          Fix Version/s 1.7.6 [ 16749 ]
          Status Open [ 1 ] Resolved [ 5 ]
          Paul King made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Hide
          Lucas Teixeira added a comment -

          I was looking on how enabling grape download feedback and got here.

          First time it is described it is without the 'groovy' prefix, but the last comment uses the groovy prefix, but "grapes" instead of "grape".

          I've found that the correct way to show is -Dgroovy.grape.report.downloads=true

          Just commenting here in case some other gets here looking for the same.

          Great work btw,

          []s,

          Show
          Lucas Teixeira added a comment - I was looking on how enabling grape download feedback and got here. First time it is described it is without the 'groovy' prefix, but the last comment uses the groovy prefix, but "grapes" instead of "grape". I've found that the correct way to show is -Dgroovy.grape.report.downloads=true Just commenting here in case some other gets here looking for the same. Great work btw, []s,
          Hide
          Ryan Nelson added a comment -

          This is a great feature, but it should be turned on by default.

          Show
          Ryan Nelson added a comment - This is a great feature, but it should be turned on by default.
          Mark Thomas made changes -
          Project Import Sun Apr 05 13:32:57 UTC 2015 [ 1428240777691 ]
          Mark Thomas made changes -
          Workflow jira [ 12732969 ] Default workflow, editable Closed status [ 12740933 ]
          Mark Thomas made changes -
          Project Import Mon Apr 06 02:11:23 UTC 2015 [ 1428286283443 ]
          Mark Thomas made changes -
          Workflow jira [ 12967410 ] Default workflow, editable Closed status [ 12975150 ]
          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Resolved Resolved
          234d 19h 37m 1 Paul King 18/Sep/10 23:45
          Resolved Resolved Closed Closed
          87d 17h 11m 1 Paul King 15/Dec/10 16:56

            People

            • Assignee:
              Paul King
              Reporter:
              Sebastian Rühl
            • Votes:
              3 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development