Uploaded image for project: 'Apache Storm'
  1. Apache Storm
  2. STORM-3245

Log viewer cleanup failes if multiple empty worker-artifact dirs

    Details

      Description

      If a drive fills up for various reasons it is possible that we can create the directory for a worker in worker-artifacts, but not able to create the worker.yaml.  If we get more than one of these on a node we get an exception like.

       

      o.a.s.d.l.u.LogCleaner logviewer-cleanup [ERROR] Exception while cleaning up old log.
      java.lang.IllegalStateException: Duplicate key ...
              at java.util.stream.Collectors.lambda$throwingMerger$0(Collectors.java:133) ~[?:1.8.0_102]
              at java.util.HashMap.merge(HashMap.java:1253) ~[?:1.8.0_102]
              at java.util.stream.Collectors.lambda$toMap$58(Collectors.java:1320) ~[?:1.8.0_102]
              at java.util.stream.ReduceOps$3ReducingSink.accept(ReduceOps.java:169) ~[?:1.8.0_102]
              at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) ~[?:1.8.0_102]
              at java.util.TreeMap$KeySpliterator.forEachRemaining(TreeMap.java:2746) ~[?:1.8.0_102]
              at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) ~[?:1.8.0_102]
              at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) ~[?:1.8.0_102]
              at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) ~[?:1.8.0_102]
              at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) ~[?:1.8.0_102]
              at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) ~[?:1.8.0_102]
              at org.apache.storm.daemon.logviewer.utils.WorkerLogs.identifyWorkerLogDirs(WorkerLogs.java:214) ~[storm-webapp-...

      After that we cannot clean up any directories any longer...

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                revans2 Robert Joseph Evans
                Reporter:
                revans2 Robert Joseph Evans
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m