Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-2191

shorten classpaths in worker and LogWriter commands

    Details

      Description

      When launching the worker daemon and its wrapping LogWriter daemon, the commands can become so long that they eclipse the default Linux limit of 4096 bytes. That results in commands that are cut off in ps output, and prevents easily inspecting the system to see even what processes are running.

      The specific scenario in which this problem can be easily triggered: running Storm on Mesos.

      Details on why it happens:
      1. using the default Mesos containerizer instead of Docker containers, which causes the storm-mesos package to be unpacked into the Mesos executor sandbox.
      2. The "expand all jars on classpath" functionality in the bin/storm.py script causes every one of the jars that storm bundles into its lib directory to be explicitly listed in the command.
        • e.g., say the mesos work dir is /var/run/mesos/work_dir/
        • and say that the original classpath argument in the supervisor cmd includes the following for the lib/ dir in the binary storm package:
          • /var/run/mesos/work_dir/slaves/2357b762-6653-4052-ab9e-f1354d78991b-S12/frameworks/20160509-084241-1086985738-5050-32231-0000/executors/STORM_TOPOLOGY_ID/runs/e6a1407e-73fd-4be4-8d00-e882117b3391/storm-mesos-0.1.7-storm0.9.6-mesos0.28.2/lib/*
        • That leads to a hugely expanded classpath argument for the LogWriter and Worker daemons that get launched:
          • /var/run/mesos/work_dir/slaves/2357b762-6653-4052-ab9e-f1354d78991b-S12/frameworks/20160509-084241-1086985738-5050-32231-0000/executors/STORM_TOPOLOGY_ID/runs/e6a1407e-73fd-4be4-8d00-e882117b3391/storm-mesos-0.1.7-storm0.9.6-mesos0.28.2/lib/asm-4.0.jar:/var/run/mesos/work_dir/slaves/2357b762-6653-4052-ab9e-f1354d78991b-S12/frameworks/20160509-084241-1086985738-5050-32231-0000/executors/STORM_TOPOLOGY_ID/runs/e6a1407e-73fd-4be4-8d00-e882117b3391/storm-mesos-0.1.7-storm0.9.6-mesos0.28.2/lib/carbonite-1.4.0.jar:...

        Attachments

          Activity

            People

            • Assignee:
              erikdw Erik Weathers
              Reporter:
              erikdw Erik Weathers
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h
                2h