Uploaded image for project: 'Flume'
  1. Flume
  2. FLUME-2379

Flume's pom files are invalid and not parseable by non-maven build tools

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: v1.4.0
    • Fix Version/s: v1.5.0
    • Component/s: Build
    • Labels:
      None

      Description

      The pom file in flume 1.4 relies on properties that are not defined outside of profiles. This is technically an invalid pom format and, while it happens to be supported by maven, is not supported by other build tools that consume poms.

      The issue is with the parent pom:
      http://repo1.maven.org/maven2/org/apache/flume/flume-parent/1.4.0/flume-parent-1.4.0.pom

      In general, AFIAK, it's not good for projects to rely on profiles for downstream consumers of build, since technically maven profiles are not considered during transitive dependency resolution.

      To fix this, default values just need to be set for properties such as `thrift.version` outside of any profiles.

      I reproduced this easily with the following build.sbt file:

      name := "Simple Project"
      
      version := "1.0"
      
      libraryDependencies += "org.apache.flume" % "flume-ng-sdk" % "1.4.0"
      

      Then you can just run `sbt compile` and it fails.

      [warn] 	::::::::::::::::::::::::::::::::::::::::::::::
      [warn] 	::          UNRESOLVED DEPENDENCIES         ::
      [warn] 	::::::::::::::::::::::::::::::::::::::::::::::
      [warn] 	:: org.apache.thrift#libthrift;${thrift.version}: not found
      [warn] 	::::::::::::::::::::::::::::::::::::::::::::::
      sbt.ResolveException: unresolved dependency: org.apache.thrift#libthrift;${thrift.version}: not found
      	at sbt.IvyActions$.sbt$IvyActions$$resolve(IvyActions.scala:213)
      	at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:122)
      	at sbt.IvyActions$$anonfun$update$1.apply(IvyActions.scala:121)
      	at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
      	at sbt.IvySbt$Module$$anonfun$withModule$1.apply(Ivy.scala:116)
      	at sbt.IvySbt$$anonfun$withIvy$1.apply(Ivy.scala:104)
      	at sbt.IvySbt.sbt$IvySbt$$action$1(Ivy.scala:51)
      	at sbt.IvySbt$$anon$3.call(Ivy.scala:60)
      	at xsbt.boot.Locks$GlobalLock.withChannel$1(Locks.scala:98)
      	at xsbt.boot.Locks$GlobalLock.xsbt$boot$Locks$GlobalLock$$withChannelRetries$1(Locks.scala:81)
      	at xsbt.boot.Locks$GlobalLock$$anonfun$withFileLock$1.apply(Locks.scala:102)
      	at xsbt.boot.Using$.withResource(Using.scala:11)
      	at xsbt.boot.Using$.apply(Using.scala:10)
      	at xsbt.boot.Locks$GlobalLock.ignoringDeadlockAvoided(Locks.scala:62)
      	at xsbt.boot.Locks$GlobalLock.withLock(Locks.scala:52)
      	at xsbt.boot.Locks$.apply0(Locks.scala:31)
      	at xsbt.boot.Locks$.apply(Locks.scala:28)
      	at sbt.IvySbt.withDefaultLogger(Ivy.scala:60)
      	at sbt.IvySbt.withIvy(Ivy.scala:101)
      	at sbt.IvySbt.withIvy(Ivy.scala:97)
      	at sbt.IvySbt$Module.withModule(Ivy.scala:116)
      	at sbt.IvyActions$.update(IvyActions.scala:121)
      	at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1161)
      	at sbt.Classpaths$$anonfun$sbt$Classpaths$$work$1$1.apply(Defaults.scala:1159)
      	at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$73.apply(Defaults.scala:1182)
      	at sbt.Classpaths$$anonfun$doWork$1$1$$anonfun$73.apply(Defaults.scala:1180)
      	at sbt.Tracked$$anonfun$lastOutput$1.apply(Tracked.scala:35)
      	at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1184)
      	at sbt.Classpaths$$anonfun$doWork$1$1.apply(Defaults.scala:1179)
      	at sbt.Tracked$$anonfun$inputChanged$1.apply(Tracked.scala:45)
      	at sbt.Classpaths$.cachedUpdate(Defaults.scala:1187)
      	at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1152)
      	at sbt.Classpaths$$anonfun$updateTask$1.apply(Defaults.scala:1130)
      	at scala.Function1$$anonfun$compose$1.apply(Function1.scala:47)
      	at sbt.$tilde$greater$$anonfun$$u2219$1.apply(TypeFunctions.scala:42)
      	at sbt.std.Transform$$anon$4.work(System.scala:64)
      	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
      	at sbt.Execute$$anonfun$submit$1$$anonfun$apply$1.apply(Execute.scala:237)
      	at sbt.ErrorHandling$.wideConvert(ErrorHandling.scala:18)
      	at sbt.Execute.work(Execute.scala:244)
      	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
      	at sbt.Execute$$anonfun$submit$1.apply(Execute.scala:237)
      	at sbt.ConcurrentRestrictions$$anon$4$$anonfun$1.apply(ConcurrentRestrictions.scala:160)
      	at sbt.CompletionService$$anon$2.call(CompletionService.scala:30)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:262)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
      	at java.lang.Thread.run(Thread.java:744)
      [error] (*:update) sbt.ResolveException: unresolved dependency: org.apache.thrift#libthrift;${thrift.version}: not found
      
      1. FLUME-2379.patch
        0.3 kB
        Hari Shreedharan

        Activity

        Hide
        hshreedharan Hari Shreedharan added a comment -

        This patch needs to be applied to trunk, flume-1.5 and flume-1.6

        Show
        hshreedharan Hari Shreedharan added a comment - This patch needs to be applied to trunk, flume-1.5 and flume-1.6
        Hide
        hshreedharan Hari Shreedharan added a comment -

        I verified that this works and pulls in the correct version of thrift in each profile. Being the RM for Flume-1.5 I am committing this.

        Show
        hshreedharan Hari Shreedharan added a comment - I verified that this works and pulls in the correct version of thrift in each profile. Being the RM for Flume-1.5 I am committing this.
        Show
        hshreedharan Hari Shreedharan added a comment - Committed here: https://git-wip-us.apache.org/repos/asf?p=flume.git;a=commit;h=73fd7bddeff11ef7af91055db1148416dca9d0b1

          People

          • Assignee:
            hshreedharan Hari Shreedharan
            Reporter:
            pwendell Patrick Wendell
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development