Uploaded image for project: 'Mesos'
  1. Mesos
  2. MESOS-3287

downloadWithHadoop tries to access Error() for a valid Try<bool>

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.24.0
    • Component/s: None
    • Labels:
    • Sprint:
      Mesosphere Sprint 17
    • Story Points:
      1

      Description

      This was reported while trying to install Hadoop / Mesos integration:

      I0818 05:36:35.058688 24428 fetcher.cpp:409] Fetcher Info: {"cache_directory":"\/tmp\/mesos\/fetch\/slaves\/20150706-075218-1611773194-5050-28439-S473\/hadoop","items":[{"action":"BYPASS_CACHE","uri":{"extract":true,"value":"hdfs:\/\/hdfs.prod:54310\/user\/ashwanth\/hadoop-with-mesos-2.6.0-cdh5.4.4.tar.gz"}}],"sandbox_directory":"\/var\/lib\/mesos\/slaves\/20150706-075218-1611773194-5050-28439-S473\/frameworks\/20150706-075218-1611773194-5050-28439-4532\/executors\/executor_Task_Tracker_4129\/runs\/c26f52d4-4055-46fa-b999-11d73f2096dd","user":"hadoop"}
      I0818 05:36:35.059806 24428 fetcher.cpp:364] Fetching URI 'hdfs://hdfs.prod:54310/user/ashwanth/hadoop-with-mesos-2.6.0-cdh5.4.4.tar.gz'
      I0818 05:36:35.059821 24428 fetcher.cpp:238] Fetching directly into the sandbox directory
      I0818 05:36:35.059835 24428 fetcher.cpp:176] Fetching URI 'hdfs://hdfs.prod:54310/user/ashwanth/hadoop-with-mesos-2.6.0-cdh5.4.4.tar.gz'
      mesos-fetcher: /tmp/mesos-build/mesos-repo/3rdparty/libprocess/3rdparty/stout/include/stout/try.hpp:90: const string& Try<T>::error() const [with T = bool; std::string = std::basic_string<char>]: Assertion `data.isNone()' failed.
      

      This is, however, a genuine bug in src/launcher/fetcher.cpp#L99:

        Try<bool> available = hdfs.available();
      
        if (available.isError() || !available.get()) {
          return Error("Skipping fetch with Hadoop Client as"
                       " Hadoop Client not available: " + available.error());
        }
      

      The root cause is that (probably) the HDFS client is not available on the slave; however, we do not error() but rather return a false result.

      The bug is exposed in the return line, where we try to retrieve available.error() (which is not there - it's just `false`).

      This was a 'latent' bug that has been exposed by (my) recent refactoring of os::shell which is used by hdfs.available() under the covers.

        Attachments

          Activity

            People

            • Assignee:
              marco-mesos Marco Massenzio
              Reporter:
              marco-mesos Marco Massenzio
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: