Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.5.0
    • Component/s: General
    • Labels:
      None

      Description

      The -Pfuse flag seems to have been removed upstream. The future support of fuse-dfs seems to be unclear (it was voted as unmaintained and removed from the default build and there are maintained alternatives, others say it still is alive ...)

      It probably still lives in the contrib folder, and can be built by ant.

      1. BIGTOP-714.patch.txt
        1 kB
        Roman Shaposhnik

        Activity

        Hide
        Roman Shaposhnik added a comment -

        Well, the important thing is – it is fixed now and I've also updated the build deps to include cmake and pkgconfig and exclude all the autotools ones.

        Show
        Roman Shaposhnik added a comment - Well, the important thing is – it is fixed now and I've also updated the build deps to include cmake and pkgconfig and exclude all the autotools ones.
        Hide
        Andrew Bayer added a comment -

        +1

        Show
        Andrew Bayer added a comment - +1
        Hide
        Erich Schubert added a comment -

        The problem for me was a missing build-dependency on pkg-config in the Debian/Ubuntu package. And a completely misleading error message.

        Show
        Erich Schubert added a comment - The problem for me was a missing build-dependency on pkg-config in the Debian/Ubuntu package. And a completely misleading error message.
        Hide
        Roman Shaposhnik added a comment -

        Thanks for reopening this. I'm attaching a patch that takes care of a few small changes that 2.0.2-alpha introduced int its layout. With this patch applied I can build hadoop (including fuse_dfs) successfully on my Ubuntu Lucid box. Please let me know if you see different results.

        Show
        Roman Shaposhnik added a comment - Thanks for reopening this. I'm attaching a patch that takes care of a few small changes that 2.0.2-alpha introduced int its layout. With this patch applied I can build hadoop (including fuse_dfs) successfully on my Ubuntu Lucid box. Please let me know if you see different results.
        Hide
        Erich Schubert added a comment -

        HDFS-3537. Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (fuse-dfs , libhdfs)
        Move libhdfs and fuse-dfs source to native subdirectories

        Show
        Erich Schubert added a comment - HDFS-3537 . Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (fuse-dfs , libhdfs) Move libhdfs and fuse-dfs source to native subdirectories
        Hide
        Erich Schubert added a comment -

        Trying to build hadoop 2.0.2-alpha, I again get the same error:

        The requested profile "fuse" could not be activated because it does not exist.

        Probably related to:

        HDFS-3537. Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (fuse-dfs , libhdfs)
        Move libhdfs and fuse-dfs source to native subdirectories

        Show
        Erich Schubert added a comment - Trying to build hadoop 2.0.2-alpha, I again get the same error: The requested profile "fuse" could not be activated because it does not exist. Probably related to: HDFS-3537 . Minor improvement reported by Colin Patrick McCabe and fixed by Colin Patrick McCabe (fuse-dfs , libhdfs) Move libhdfs and fuse-dfs source to native subdirectories
        Hide
        Erich Schubert added a comment -

        This is an issue only with 0.23.3, and odd numbered release of Hadoop. 2.0.1-alpha is ok, too, and there aren't any visible changes to be seen. Apparently all that "drop fuse" discussion was solved differently.

        Show
        Erich Schubert added a comment - This is an issue only with 0.23.3, and odd numbered release of Hadoop. 2.0.1-alpha is ok, too, and there aren't any visible changes to be seen. Apparently all that "drop fuse" discussion was solved differently.
        Hide
        Roman Shaposhnik added a comment -

        I was under the impression that 0.23.3 is the latest Hadoop release (being released 2 months after 2.0.1-alpha).

        Welcome to the world of Hadoop version numbers – where releases and versions don't make much sense (still). Now, to be fair the whole renumbering to 1.X and 2.X happened precisely because of the mess that uses to be.

        Anyway, at the moment the only two actively developed Hadoop branches are Hadoop 1.X and Hadoop 2.X everything else is maintained for particular use cases with release managers essentially cherry-picking fixes ONLY for the things they care about. IOW, all of the bug fixes, etc. go either into 1.X or 2.X. You're really way better off using one of those branches. Don't be confused by the release numbers since in Hadoop (actually all of the Apache) it is not uncommon to have releases running in parallel.

        Can you build the latest hadoop with bigtop, not 2.0.0-alpha as in bigtop trunk?

        The latest released Hadoop in 2.X seriues is Hadoop 2.0.1-aplha and we are likely to target 2.0.2-alpha for the Bigtop 0.5.0. I believe neither has this issue. If you're interested in Hadoop 1.X then the latest release one is Hadoop 1.0.3 which also doesn't seem to suffer from this problem.

        Once again: anybody who cares about 0.23.X should take it up with the current release manager (Bobby Evans from Yahoo!)

        Show
        Roman Shaposhnik added a comment - I was under the impression that 0.23.3 is the latest Hadoop release (being released 2 months after 2.0.1-alpha). Welcome to the world of Hadoop version numbers – where releases and versions don't make much sense (still). Now, to be fair the whole renumbering to 1.X and 2.X happened precisely because of the mess that uses to be. Anyway, at the moment the only two actively developed Hadoop branches are Hadoop 1.X and Hadoop 2.X everything else is maintained for particular use cases with release managers essentially cherry-picking fixes ONLY for the things they care about. IOW, all of the bug fixes, etc. go either into 1.X or 2.X. You're really way better off using one of those branches. Don't be confused by the release numbers since in Hadoop (actually all of the Apache) it is not uncommon to have releases running in parallel. Can you build the latest hadoop with bigtop, not 2.0.0-alpha as in bigtop trunk? The latest released Hadoop in 2.X seriues is Hadoop 2.0.1-aplha and we are likely to target 2.0.2-alpha for the Bigtop 0.5.0. I believe neither has this issue. If you're interested in Hadoop 1.X then the latest release one is Hadoop 1.0.3 which also doesn't seem to suffer from this problem. Once again: anybody who cares about 0.23.X should take it up with the current release manager (Bobby Evans from Yahoo!)
        Hide
        Erich Schubert added a comment -

        I was under the impression that 0.23.3 is the latest Hadoop release (being released 2 months after 2.0.1-alpha). In fact, the 2.0.0-alpha release notes identify that release as 0.23.1:

        http://hadoop.apache.org/docs/r2.0.0-alpha/hadoop-project-dist/hadoop-common/releasenotes.html

        So if this is 0.23.1, shouldn't 0.23.3 be newer, just not yet rebranded as 2.0.2-alpha (or 2.0.1-beta?). And in SVN, they all seem to go with 3.0.0-SNAPSHOT or so already... Versionitis, as pushed by IOS/Chrome/Firefox I guess. There is an internal version number, and a marketing version number. This is at least he impression I had.

        The apache web site somewhere also says:

        .bq 0.23.X - simmilar to 2.X.X but missing NN HA.

        I do not need NN HA, as I currently have only three nodes.

        Anyway, if I recall correctly, I had the same issue when trying to build Hadoop "trunk", that -Pfuse was removed. So I belive this was a decision done at upstream Hadoop, and that's why I filed this with BIGTOP.

        Can you build the latest hadoop with bigtop, not 2.0.0-alpha as in bigtop trunk?

        Show
        Erich Schubert added a comment - I was under the impression that 0.23.3 is the latest Hadoop release (being released 2 months after 2.0.1-alpha). In fact, the 2.0.0-alpha release notes identify that release as 0.23.1: http://hadoop.apache.org/docs/r2.0.0-alpha/hadoop-project-dist/hadoop-common/releasenotes.html So if this is 0.23.1, shouldn't 0.23.3 be newer, just not yet rebranded as 2.0.2-alpha (or 2.0.1-beta?). And in SVN, they all seem to go with 3.0.0-SNAPSHOT or so already... Versionitis, as pushed by IOS/Chrome/Firefox I guess. There is an internal version number, and a marketing version number. This is at least he impression I had. The apache web site somewhere also says: .bq 0.23.X - simmilar to 2.X.X but missing NN HA. I do not need NN HA, as I currently have only three nodes. Anyway, if I recall correctly, I had the same issue when trying to build Hadoop "trunk", that -Pfuse was removed. So I belive this was a decision done at upstream Hadoop, and that's why I filed this with BIGTOP. Can you build the latest hadoop with bigtop, not 2.0.0-alpha as in bigtop trunk?
        Hide
        Roman Shaposhnik added a comment -

        A couple of point: Hadoop 0.23.0 has been completely superseded by Hadoop 2.X. Is there a reason you're still looking at 0.23?

        As for the latest Hadoop version (which should be 2.0.2-alpha-SNAPSHOT at the moment) – thanks for point it out – we'll definitely take a look at it (although the fix and the JIRA will have to go to the upstream Hadoop community)

        Show
        Roman Shaposhnik added a comment - A couple of point: Hadoop 0.23.0 has been completely superseded by Hadoop 2.X. Is there a reason you're still looking at 0.23? As for the latest Hadoop version (which should be 2.0.2-alpha-SNAPSHOT at the moment) – thanks for point it out – we'll definitely take a look at it (although the fix and the JIRA will have to go to the upstream Hadoop community)
        Hide
        Erich Schubert added a comment -

        Trunk and 0.4.0 release as far as I can tell use Hadoop 2.0.0 (-alpha? = 0.23.0?) and not 0.23.3. When I tried to build the latest Hadoop with Bigtop, I get the error that -Pfuse is unknown. Try building the latest Hadoop version.

        https://svn.apache.org/repos/asf/incubator/bigtop/trunk/bigtop.mk

        says it is still using "2.0.0". To me it looked as if the fuse component got demoted in the newer releases, and disappeared from pom.xml

        Show
        Erich Schubert added a comment - Trunk and 0.4.0 release as far as I can tell use Hadoop 2.0.0 (-alpha? = 0.23.0?) and not 0.23.3. When I tried to build the latest Hadoop with Bigtop, I get the error that -Pfuse is unknown. Try building the latest Hadoop version. https://svn.apache.org/repos/asf/incubator/bigtop/trunk/bigtop.mk says it is still using "2.0.0". To me it looked as if the fuse component got demoted in the newer releases, and disappeared from pom.xml
        Hide
        Roman Shaposhnik added a comment -

        Erich, could you please elaborate as what the issue here is with Bigtop? I just tried our nightly trunk build and our 0.4.0 release and the following works:

        $ sudo hadoop-fuse-dfs dfs://localhost:8020   /mnt
        

        what is the issue that you're trying to address?

        Show
        Roman Shaposhnik added a comment - Erich, could you please elaborate as what the issue here is with Bigtop? I just tried our nightly trunk build and our 0.4.0 release and the following works: $ sudo hadoop-fuse-dfs dfs://localhost:8020 /mnt what is the issue that you're trying to address?

          People

          • Assignee:
            Roman Shaposhnik
            Reporter:
            Erich Schubert
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development