Uploaded image for project: 'Apache RocketMQ'
  1. Apache RocketMQ
  2. ROCKETMQ-31

Deletion needed for `$HOME/rmq_bk_gc.log` in `bin/mqbroker`

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.0.0-incubating
    • Component/s: None
    • Labels:
      None

      Description

      Going through "Quick Start" manual, I got warning message when I start bin/mqbroker as below:

      cp: /Users/kailai/rmq_bk_gc.log: No such file or directory
      

      The error is caused by the following code in bin/mqbroker:

      rm -f $HOME/rmq_bk_gc.log.bac
      cp $HOME/rmq_bk_gc.log $HOME/rmq_bk_gc.log.bac
      

      Since the GC log has been set to /dev/shm/mq_gc_%p.log due to bin/runbroker.sh Line 41, the operation for $HOME/rmq_bk_gc.log in bin/mqbroker should be deleted.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user iskl opened a pull request:

          https://github.com/apache/incubator-rocketmq/pull/29

          ROCKETMQ-31 Fix "No such file or directory" warning for $HOME/rmq_b…

          The PR is to resolve the issue ROCKETMQ-31(https://issues.apache.org/jira/browse/ROCKETMQ-31)

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/iskl/incubator-rocketmq ROCKETMQ-31

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/incubator-rocketmq/pull/29.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #29


          commit 619188cfa8c1e5d5ad21880154ab4fbac8fdb9d1
          Author: kailai.skl <kailai.skl@alipay.com>
          Date: 2017-01-05T11:10:13Z

          ROCKETMQ-31 Fix "No such file or directory" warning for $HOME/rmq_bk_gc.log


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user iskl opened a pull request: https://github.com/apache/incubator-rocketmq/pull/29 ROCKETMQ-31 Fix "No such file or directory" warning for $HOME/rmq_b… The PR is to resolve the issue ROCKETMQ-31 ( https://issues.apache.org/jira/browse/ROCKETMQ-31 ) You can merge this pull request into a Git repository by running: $ git pull https://github.com/iskl/incubator-rocketmq ROCKETMQ-31 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rocketmq/pull/29.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #29 commit 619188cfa8c1e5d5ad21880154ab4fbac8fdb9d1 Author: kailai.skl <kailai.skl@alipay.com> Date: 2017-01-05T11:10:13Z ROCKETMQ-31 Fix "No such file or directory" warning for $HOME/rmq_bk_gc.log
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user shroman commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/29

          I would fix `-Xloggc:/dev/shm/mq_gc_%p.log` to point to `$HOME` instead of removing these two lines

          Show
          githubbot ASF GitHub Bot added a comment - Github user shroman commented on the issue: https://github.com/apache/incubator-rocketmq/pull/29 I would fix `-Xloggc:/dev/shm/mq_gc_%p.log` to point to `$HOME` instead of removing these two lines
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user iskl commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/29

          Dear @shroman , thanks for your suggestion

          In my concept, broker thread is *time sensitive. When broker thread in a MQ broker is in I/O operation and GC occurs, an **I/O competition* appears between broker thread and GC thread. It may bring *long time latency to broker thread*.

          Linux distributions based on the 2.6 kernel and later offer `/dev/shm` as shared memory in the form of a *RAM disk* [(link)](https://en.wikipedia.org/wiki/Shared_memory). Pointing GC log to `/dev/shm` can avoid I/O competition between broker thread and GC thread.

          In conclusion, I tend to use `-Xloggc:/dev/shm/mq_gc_%p.log`

          Show
          githubbot ASF GitHub Bot added a comment - Github user iskl commented on the issue: https://github.com/apache/incubator-rocketmq/pull/29 Dear @shroman , thanks for your suggestion In my concept, broker thread is * time sensitive . When broker thread in a MQ broker is in I/O operation and GC occurs, an **I/O competition * appears between broker thread and GC thread. It may bring * long time latency to broker thread *. Linux distributions based on the 2.6 kernel and later offer `/dev/shm` as shared memory in the form of a * RAM disk * [(link)] ( https://en.wikipedia.org/wiki/Shared_memory ). Pointing GC log to `/dev/shm` can avoid I/O competition between broker thread and GC thread. In conclusion, I tend to use `-Xloggc:/dev/shm/mq_gc_%p.log`
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user shroman commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/29

          yes, I see, you want to reduce disk I/O by placing the file into `/dev/shm`.

          Well, GC logging has a minimal overhead. And to get a high-performance broker, I would tune my JVM to get smaller GC pauses, which has much larger impact on the overall performance.

          Also, is it ok not to persist your GC logs? Long-term logging is essential for analysis of the brokers' performance.

          Show
          githubbot ASF GitHub Bot added a comment - Github user shroman commented on the issue: https://github.com/apache/incubator-rocketmq/pull/29 yes, I see, you want to reduce disk I/O by placing the file into `/dev/shm`. Well, GC logging has a minimal overhead. And to get a high-performance broker, I would tune my JVM to get smaller GC pauses, which has much larger impact on the overall performance. Also, is it ok not to persist your GC logs? Long-term logging is essential for analysis of the brokers' performance.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zhouxinyu commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/29

          It's our mistake. As @iskl says, we print gc logs to tmpfs in order to eliminate long time latency brought by disk I/O competition.

          IMO, we can merge this PR at current state, and open another issue to discuss where to place GC logs. How about your advise? @shroman -

          Show
          githubbot ASF GitHub Bot added a comment - Github user zhouxinyu commented on the issue: https://github.com/apache/incubator-rocketmq/pull/29 It's our mistake. As @iskl says, we print gc logs to tmpfs in order to eliminate long time latency brought by disk I/O competition. IMO, we can merge this PR at current state, and open another issue to discuss where to place GC logs. How about your advise? @shroman -
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user shroman commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/29

          @zhouxinyu @iskl's concerns are valid indeed.
          My major concern is the logs can be lost when the system crashes. I think it's a good idea to discuss it in another JIRA issue.
          I will create a JIRA issue, if it's ok with you.

          Show
          githubbot ASF GitHub Bot added a comment - Github user shroman commented on the issue: https://github.com/apache/incubator-rocketmq/pull/29 @zhouxinyu @iskl's concerns are valid indeed. My major concern is the logs can be lost when the system crashes. I think it's a good idea to discuss it in another JIRA issue. I will create a JIRA issue, if it's ok with you.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zhouxinyu commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/29

          @shroman it's ok surely.
          Please @lizhanhui @stevenschew help review this pr.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zhouxinyu commented on the issue: https://github.com/apache/incubator-rocketmq/pull/29 @shroman it's ok surely. Please @lizhanhui @stevenschew help review this pr.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user stevenschew commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/29

          it's good idea, review ok

          Show
          githubbot ASF GitHub Bot added a comment - Github user stevenschew commented on the issue: https://github.com/apache/incubator-rocketmq/pull/29 it's good idea, review ok
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user shroman commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/29

          @zhouxinyu @stevenschew Thanks, guys, https://issues.apache.org/jira/browse/ROCKETMQ-36 is created to consider the improvement.

          Show
          githubbot ASF GitHub Bot added a comment - Github user shroman commented on the issue: https://github.com/apache/incubator-rocketmq/pull/29 @zhouxinyu @stevenschew Thanks, guys, https://issues.apache.org/jira/browse/ROCKETMQ-36 is created to consider the improvement.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit f93605755f83b4a992b38538c33c1348517699a6 in incubator-rocketmq's branch refs/heads/master from Kailai Shao
          [ https://git-wip-us.apache.org/repos/asf?p=incubator-rocketmq.git;h=f936057 ]

          ROCKETMQ-31 Fix 'No such file or directory' warning for /Users/zhouxinyu/rmq_bk_gc.log, closes apache/incubator-rocketmq#29

          Show
          jira-bot ASF subversion and git services added a comment - Commit f93605755f83b4a992b38538c33c1348517699a6 in incubator-rocketmq's branch refs/heads/master from Kailai Shao [ https://git-wip-us.apache.org/repos/asf?p=incubator-rocketmq.git;h=f936057 ] ROCKETMQ-31 Fix 'No such file or directory' warning for /Users/zhouxinyu/rmq_bk_gc.log, closes apache/incubator-rocketmq#29
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/incubator-rocketmq/pull/29

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/incubator-rocketmq/pull/29

            People

            • Assignee:
              vongosling vongosling
              Reporter:
              kailai Kailai Shao
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development