Details

      Description

      So far, RocketMQ only support message filtering feature by `TAG`, but one message only can own one tag, this is too limited to meet complex business requirements.

      So, we want to define and implement a reasonable filter language based on a subset of the SQL 92 expression syntax to support customized message filtering.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user Jaskey commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          @vsair

          Thanks for your clarification. I am just trying to go through the main logic of sql filter.

          For the point of #2, where is that code logic ? I search for usage of `getExpectConsumerNumUseFilter` but I find that it's only used to create a new bloom filter instance when starting , then how do broker detect that the configuration is changed? Are we saying the same thing.

          What I am concerned is that, if we change any configuration about the bloom filter, will broker works as usual to filter the expected messages as it did before.

          Show
          githubbot ASF GitHub Bot added a comment - Github user Jaskey commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 @vsair Thanks for your clarification. I am just trying to go through the main logic of sql filter. For the point of #2, where is that code logic ? I search for usage of `getExpectConsumerNumUseFilter` but I find that it's only used to create a new bloom filter instance when starting , then how do broker detect that the configuration is changed? Are we saying the same thing. What I am concerned is that, if we change any configuration about the bloom filter, will broker works as usual to filter the expected messages as it did before.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vsair commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          Hi, Jaskey;

          Thanks for your question.

          1. It will decode the properties to do calculation at that case.
          2. When they are changed, the consumer's subscription will be ignored when
          starting, so the results int bloom filter also will be ignored. That is
          mean server will do calculation again and not update the results already
          existed in Store.

          Thanks,
          Vsair

          On Mon, Jun 12, 2017 at 2:47 PM, Jaskey <notifications@github.com> wrote:

          > @vsair <https://github.com/vsair>
          >
          > I have two things to confirms,
          >
          > 1.
          >
          > if the old messages which exists before consumer subscribe with the
          > filter expression must be filtered by decodng properties to do calculation ?
          > 2.
          >
          > `expectConsumerNumUseFilter = 32` and `maxErrorRateOfBloomFilter = 20` can be configured and changed, what if I change this value and restart, does the calculated results in the ConsumeQueueExt updated accordingly?
          >
          >
          > —
          > You are receiving this because you were mentioned.
          > Reply to this email directly, view it on GitHub
          > <https://github.com/apache/incubator-rocketmq/pull/82#issuecomment-307704104>,
          > or mute the thread
          > <https://github.com/notifications/unsubscribe-auth/ACeujdZf1ZNbu-MycGfHW_kbIuCCvUE3ks5sDN8ZgaJpZM4Mk0bH>
          > .
          >

          Show
          githubbot ASF GitHub Bot added a comment - Github user vsair commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 Hi, Jaskey; Thanks for your question. 1. It will decode the properties to do calculation at that case. 2. When they are changed, the consumer's subscription will be ignored when starting, so the results int bloom filter also will be ignored. That is mean server will do calculation again and not update the results already existed in Store. Thanks, Vsair On Mon, Jun 12, 2017 at 2:47 PM, Jaskey <notifications@github.com> wrote: > @vsair < https://github.com/vsair > > > I have two things to confirms, > > 1. > > if the old messages which exists before consumer subscribe with the > filter expression must be filtered by decodng properties to do calculation ? > 2. > > `expectConsumerNumUseFilter = 32` and `maxErrorRateOfBloomFilter = 20` can be configured and changed, what if I change this value and restart, does the calculated results in the ConsumeQueueExt updated accordingly? > > > — > You are receiving this because you were mentioned. > Reply to this email directly, view it on GitHub > < https://github.com/apache/incubator-rocketmq/pull/82#issuecomment-307704104 >, > or mute the thread > < https://github.com/notifications/unsubscribe-auth/ACeujdZf1ZNbu-MycGfHW_kbIuCCvUE3ks5sDN8ZgaJpZM4Mk0bH > > . >
          Hide
          zander dongeforever added a comment -

          Github user asfgit closed the pull request at:
          https://github.com/apache/incubator-rocketmq-site/pull/11

          Show
          zander dongeforever added a comment - Github user asfgit closed the pull request at: https://github.com/apache/incubator-rocketmq-site/pull/11
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/incubator-rocketmq-site/pull/11

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/incubator-rocketmq-site/pull/11
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user vsair opened a pull request:

          https://github.com/apache/incubator-rocketmq-site/pull/11

          ROCKETMQ-121Docs of filtering messages based on SQL92

          Docs of filtering messages based on SQL92.

          PR, already merged to develop:
          https://github.com/apache/incubator-rocketmq/pull/82

          Issue:
          https://issues.apache.org/jira/browse/ROCKETMQ-121

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/vsair/incubator-rocketmq-site ROCKETMQ-121

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/incubator-rocketmq-site/pull/11.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #11


          commit 54f2b985ef7b76e2502428332bcaee5c0b758a2b
          Author: vsair <liuxuedee@gmail.com>
          Date: 2017-04-26T08:36:48Z

          ROCKETMQ-121Docs of filtering messages based on SQL92


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user vsair opened a pull request: https://github.com/apache/incubator-rocketmq-site/pull/11 ROCKETMQ-121 Docs of filtering messages based on SQL92 Docs of filtering messages based on SQL92. PR, already merged to develop: https://github.com/apache/incubator-rocketmq/pull/82 Issue: https://issues.apache.org/jira/browse/ROCKETMQ-121 You can merge this pull request into a Git repository by running: $ git pull https://github.com/vsair/incubator-rocketmq-site ROCKETMQ-121 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/incubator-rocketmq-site/pull/11.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #11 commit 54f2b985ef7b76e2502428332bcaee5c0b758a2b Author: vsair <liuxuedee@gmail.com> Date: 2017-04-26T08:36:48Z ROCKETMQ-121 Docs of filtering messages based on SQL92
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vsair closed the pull request at:

          https://github.com/apache/incubator-rocketmq/pull/82

          Show
          githubbot ASF GitHub Bot added a comment - Github user vsair closed the pull request at: https://github.com/apache/incubator-rocketmq/pull/82
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 58f1574b28bf8bf18a795036545c7a700437ed0b in incubator-rocketmq's branch refs/heads/develop from Eric Liu
          [ https://git-wip-us.apache.org/repos/asf?p=incubator-rocketmq.git;h=58f1574 ]

          ROCKETMQ-121Support message filtering based on SQL92 closes apache/incubator-rocketmq#82

          Show
          jira-bot ASF subversion and git services added a comment - Commit 58f1574b28bf8bf18a795036545c7a700437ed0b in incubator-rocketmq's branch refs/heads/develop from Eric Liu [ https://git-wip-us.apache.org/repos/asf?p=incubator-rocketmq.git;h=58f1574 ] ROCKETMQ-121 Support message filtering based on SQL92 closes apache/incubator-rocketmq#82
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user dongeforever commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          It seems that this PR could be merged now. @vongosling @zhouxinyu @lizhanhui

          Show
          githubbot ASF GitHub Bot added a comment - Github user dongeforever commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 It seems that this PR could be merged now. @vongosling @zhouxinyu @lizhanhui
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vsair commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          @zhouxinyu Conflicts have been resolved. Thanks.

          Show
          githubbot ASF GitHub Bot added a comment - Github user vsair commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 @zhouxinyu Conflicts have been resolved. Thanks.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/11175865/badge)](https://coveralls.io/builds/11175865)

          Coverage increased (+3.3%) to 37.892% when pulling *09dca7851d7a85e4e609a5f34d6abd91dc092b91 on vsair:ROCKETMQ-121* into *42f78c281cbeb5072b04eaf03b1a8059b8d281a7 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/11175865/badge)](https://coveralls.io/builds/11175865 ) Coverage increased (+3.3%) to 37.892% when pulling * 09dca7851d7a85e4e609a5f34d6abd91dc092b91 on vsair: ROCKETMQ-121 * into * 42f78c281cbeb5072b04eaf03b1a8059b8d281a7 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/11175865/badge)](https://coveralls.io/builds/11175865)

          Coverage increased (+3.3%) to 37.892% when pulling *09dca7851d7a85e4e609a5f34d6abd91dc092b91 on vsair:ROCKETMQ-121* into *42f78c281cbeb5072b04eaf03b1a8059b8d281a7 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/11175865/badge)](https://coveralls.io/builds/11175865 ) Coverage increased (+3.3%) to 37.892% when pulling * 09dca7851d7a85e4e609a5f34d6abd91dc092b91 on vsair: ROCKETMQ-121 * into * 42f78c281cbeb5072b04eaf03b1a8059b8d281a7 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/11175865/badge)](https://coveralls.io/builds/11175865)

          Coverage increased (+3.3%) to 37.892% when pulling *09dca7851d7a85e4e609a5f34d6abd91dc092b91 on vsair:ROCKETMQ-121* into *42f78c281cbeb5072b04eaf03b1a8059b8d281a7 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/11175865/badge)](https://coveralls.io/builds/11175865 ) Coverage increased (+3.3%) to 37.892% when pulling * 09dca7851d7a85e4e609a5f34d6abd91dc092b91 on vsair: ROCKETMQ-121 * into * 42f78c281cbeb5072b04eaf03b1a8059b8d281a7 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user lizhanhui commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          @zhouxinyu I have checked the major data flow and it looks good to me. I have not scrutinized the changes line by line yet. You may merge it first and we may discuss potential issues hereafter.

          Show
          githubbot ASF GitHub Bot added a comment - Github user lizhanhui commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 @zhouxinyu I have checked the major data flow and it looks good to me. I have not scrutinized the changes line by line yet. You may merge it first and we may discuss potential issues hereafter.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user zhouxinyu commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          @lizhanhui , Do you have any other thoughts about this PR? If no, I will merge this PR soon.

          And please @vsair help resolve the conflicting files.

          Show
          githubbot ASF GitHub Bot added a comment - Github user zhouxinyu commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 @lizhanhui , Do you have any other thoughts about this PR? If no, I will merge this PR soon. And please @vsair help resolve the conflicting files.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vsair commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          @vongosling yeah, only server's modules have guava dependency, client doesn't.

          Show
          githubbot ASF GitHub Bot added a comment - Github user vsair commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 @vongosling yeah, only server's modules have guava dependency, client doesn't.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          please @lizhanhui @shroman help to review this great PR

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 please @lizhanhui @shroman help to review this great PR
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          I think only servtool‘s module dependency, so there is no transitive dependency pollution on sdk, right?

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 I think only servtool‘s module dependency, so there is no transitive dependency pollution on sdk, right?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r111732484

          — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/BloomFilter.java —
          @@ -0,0 +1,332 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.filter.util;
          +
          +import java.util.Arrays;
          +
          +/**
          + * Simple implement of bloom filter.
          + */
          +public class BloomFilter {
          — End diff –

          Great, murmur3_128

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r111732484 — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/BloomFilter.java — @@ -0,0 +1,332 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.filter.util; + +import java.util.Arrays; + +/** + * Simple implement of bloom filter. + */ +public class BloomFilter { — End diff – Great, murmur3_128
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user lizhanhui commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          Will look into this PR today.

          Show
          githubbot ASF GitHub Bot added a comment - Github user lizhanhui commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 Will look into this PR today.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          @shroman @lizhanhui What's your opinion about this feature

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 @shroman @lizhanhui What's your opinion about this feature
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/11031179/badge)](https://coveralls.io/builds/11031179)

          Coverage increased (+3.4%) to 35.262% when pulling *24f6ada53a4b3f0612461a0c69dc49b2508104b0 on vsair:ROCKETMQ-121* into *45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/11031179/badge)](https://coveralls.io/builds/11031179 ) Coverage increased (+3.4%) to 35.262% when pulling * 24f6ada53a4b3f0612461a0c69dc49b2508104b0 on vsair: ROCKETMQ-121 * into * 45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/11031179/badge)](https://coveralls.io/builds/11031179)

          Coverage increased (+3.4%) to 35.262% when pulling *24f6ada53a4b3f0612461a0c69dc49b2508104b0 on vsair:ROCKETMQ-121* into *45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/11031179/badge)](https://coveralls.io/builds/11031179 ) Coverage increased (+3.4%) to 35.262% when pulling * 24f6ada53a4b3f0612461a0c69dc49b2508104b0 on vsair: ROCKETMQ-121 * into * 45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/11031179/badge)](https://coveralls.io/builds/11031179)

          Coverage increased (+3.4%) to 35.262% when pulling *24f6ada53a4b3f0612461a0c69dc49b2508104b0 on vsair:ROCKETMQ-121* into *45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/11031179/badge)](https://coveralls.io/builds/11031179 ) Coverage increased (+3.4%) to 35.262% when pulling * 24f6ada53a4b3f0612461a0c69dc49b2508104b0 on vsair: ROCKETMQ-121 * into * 45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vsair commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r110845437

          — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/ConsumerFilterData.java —
          @@ -0,0 +1,180 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.broker.filter;
          +
          +import org.apache.rocketmq.filter.expression.Expression;
          +import org.apache.rocketmq.filter.util.BloomFilterData;
          +
          +/**
          + * Filter data of consumer.
          + */
          +public class ConsumerFilterData {
          +
          + private String consumerGroup;
          + private String topic;
          + private String expression;
          + private String expressionType;
          + private transient Expression compiledExpression;
          + private long bornTime;
          + private long deadTime = 0;
          + private BloomFilterData bloomFilterData;
          + private long clientVersion;
          +
          + public boolean isDead()

          { + return this.deadTime >= this.bornTime; + }

          +
          + public long deadHowLong() {
          + if (isDead())

          { + return System.currentTimeMillis() - getDeadTime(); + }

          + return -1;
          + }
          +
          + /**
          + * Check this filter data has been used to calculate bit map when msg was stored in server.
          + *
          + * @param msgStoreTime
          + * @return
          + */
          + public boolean isMsgInLive(long msgStoreTime)

          { + return msgStoreTime > getBornTime(); + }

          +
          + public String getConsumerGroup()

          { + return consumerGroup; + }

          +
          + public void setConsumerGroup(final String consumerGroup)

          { + this.consumerGroup = consumerGroup; + }

          +
          + public String getTopic()

          { + return topic; + }

          +
          + public void setTopic(final String topic)

          { + this.topic = topic; + }

          +
          + public String getExpression()

          { + return expression; + }

          +
          + public void setExpression(final String expression)

          { + this.expression = expression; + }

          +
          + public String getExpressionType()

          { + return expressionType; + }

          +
          + public void setExpressionType(final String expressionType)

          { + this.expressionType = expressionType; + }

          +
          + public Expression getCompiledExpression()

          { + return compiledExpression; + }

          +
          + public void setCompiledExpression(final Expression compiledExpression)

          { + this.compiledExpression = compiledExpression; + }

          +
          + public long getBornTime()

          { + return bornTime; + }

          +
          + public void setBornTime(final long bornTime)

          { + this.bornTime = bornTime; + }

          +
          + public long getDeadTime()

          { + return deadTime; + }

          +
          + public void setDeadTime(final long deadTime)

          { + this.deadTime = deadTime; + }

          +
          + public BloomFilterData getBloomFilterData()

          { + return bloomFilterData; + }

          +
          + public void setBloomFilterData(final BloomFilterData bloomFilterData)

          { + this.bloomFilterData = bloomFilterData; + }

          +
          + public long getClientVersion()

          { + return clientVersion; + }

          +
          + public void setClientVersion(long clientVersion)

          { + this.clientVersion = clientVersion; + }

          +
          + @Override
          + public boolean equals(Object o) {
          — End diff –

          Already replaced by EqualsBuilder, ToStringBuilder and HashCodeBuilder.

          Show
          githubbot ASF GitHub Bot added a comment - Github user vsair commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r110845437 — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/ConsumerFilterData.java — @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.broker.filter; + +import org.apache.rocketmq.filter.expression.Expression; +import org.apache.rocketmq.filter.util.BloomFilterData; + +/** + * Filter data of consumer. + */ +public class ConsumerFilterData { + + private String consumerGroup; + private String topic; + private String expression; + private String expressionType; + private transient Expression compiledExpression; + private long bornTime; + private long deadTime = 0; + private BloomFilterData bloomFilterData; + private long clientVersion; + + public boolean isDead() { + return this.deadTime >= this.bornTime; + } + + public long deadHowLong() { + if (isDead()) { + return System.currentTimeMillis() - getDeadTime(); + } + return -1; + } + + /** + * Check this filter data has been used to calculate bit map when msg was stored in server. + * + * @param msgStoreTime + * @return + */ + public boolean isMsgInLive(long msgStoreTime) { + return msgStoreTime > getBornTime(); + } + + public String getConsumerGroup() { + return consumerGroup; + } + + public void setConsumerGroup(final String consumerGroup) { + this.consumerGroup = consumerGroup; + } + + public String getTopic() { + return topic; + } + + public void setTopic(final String topic) { + this.topic = topic; + } + + public String getExpression() { + return expression; + } + + public void setExpression(final String expression) { + this.expression = expression; + } + + public String getExpressionType() { + return expressionType; + } + + public void setExpressionType(final String expressionType) { + this.expressionType = expressionType; + } + + public Expression getCompiledExpression() { + return compiledExpression; + } + + public void setCompiledExpression(final Expression compiledExpression) { + this.compiledExpression = compiledExpression; + } + + public long getBornTime() { + return bornTime; + } + + public void setBornTime(final long bornTime) { + this.bornTime = bornTime; + } + + public long getDeadTime() { + return deadTime; + } + + public void setDeadTime(final long deadTime) { + this.deadTime = deadTime; + } + + public BloomFilterData getBloomFilterData() { + return bloomFilterData; + } + + public void setBloomFilterData(final BloomFilterData bloomFilterData) { + this.bloomFilterData = bloomFilterData; + } + + public long getClientVersion() { + return clientVersion; + } + + public void setClientVersion(long clientVersion) { + this.clientVersion = clientVersion; + } + + @Override + public boolean equals(Object o) { — End diff – Already replaced by EqualsBuilder, ToStringBuilder and HashCodeBuilder.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vsair commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r110845213

          — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/BloomFilter.java —
          @@ -0,0 +1,332 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.filter.util;
          +
          +import java.util.Arrays;
          +
          +/**
          + * Simple implement of bloom filter.
          + */
          +public class BloomFilter {
          — End diff –

          Hi, I replaced HashAlgorithm to Guava Hashing.murmur3_128. Most classes of Guava's BloomFilter are private and final, which could not be included by RocketMQ, such as BitArray saved in store.So I didn't use it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user vsair commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r110845213 — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/BloomFilter.java — @@ -0,0 +1,332 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.filter.util; + +import java.util.Arrays; + +/** + * Simple implement of bloom filter. + */ +public class BloomFilter { — End diff – Hi, I replaced HashAlgorithm to Guava Hashing.murmur3_128. Most classes of Guava's BloomFilter are private and final, which could not be included by RocketMQ, such as BitArray saved in store.So I didn't use it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/11028471/badge)](https://coveralls.io/builds/11028471)

          Coverage increased (+3.7%) to 35.602% when pulling *069256828c512370c2b2aeb48d64481d5c6d1215 on vsair:ROCKETMQ-121* into *45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/11028471/badge)](https://coveralls.io/builds/11028471 ) Coverage increased (+3.7%) to 35.602% when pulling * 069256828c512370c2b2aeb48d64481d5c6d1215 on vsair: ROCKETMQ-121 * into * 45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/11028471/badge)](https://coveralls.io/builds/11028471)

          Coverage increased (+3.7%) to 35.602% when pulling *069256828c512370c2b2aeb48d64481d5c6d1215 on vsair:ROCKETMQ-121* into *45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/11028471/badge)](https://coveralls.io/builds/11028471 ) Coverage increased (+3.7%) to 35.602% when pulling * 069256828c512370c2b2aeb48d64481d5c6d1215 on vsair: ROCKETMQ-121 * into * 45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/11028471/badge)](https://coveralls.io/builds/11028471)

          Coverage increased (+3.7%) to 35.602% when pulling *069256828c512370c2b2aeb48d64481d5c6d1215 on vsair:ROCKETMQ-121* into *45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/11028471/badge)](https://coveralls.io/builds/11028471 ) Coverage increased (+3.7%) to 35.602% when pulling * 069256828c512370c2b2aeb48d64481d5c6d1215 on vsair: ROCKETMQ-121 * into * 45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vsair commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r109138716

          — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/HashAlgorithm.java —
          @@ -0,0 +1,239 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.filter.util;
          +
          +import java.io.UnsupportedEncodingException;
          +import java.security.MessageDigest;
          +import java.security.NoSuchAlgorithmException;
          +import java.util.zip.CRC32;
          +
          +/**
          + * Note that all hash
          + * algorithms return 64-bits of hash, but only the lower 32-bits are
          + * significant. This allows a positive 32-bit number to be returned for all
          + * cases.
          + * <br>Don't change the order of algorithms, add new algorithm to last if you want.
          + */
          — End diff –

          Ok, I'll check whether the third party's jar could meet the scene.

          Show
          githubbot ASF GitHub Bot added a comment - Github user vsair commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r109138716 — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/HashAlgorithm.java — @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.filter.util; + +import java.io.UnsupportedEncodingException; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.util.zip.CRC32; + +/** + * Note that all hash + * algorithms return 64-bits of hash, but only the lower 32-bits are + * significant. This allows a positive 32-bit number to be returned for all + * cases. + * <br>Don't change the order of algorithms, add new algorithm to last if you want. + */ — End diff – Ok, I'll check whether the third party's jar could meet the scene.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108884189

          — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/ConsumerFilterData.java —
          @@ -0,0 +1,180 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.broker.filter;
          +
          +import org.apache.rocketmq.filter.expression.Expression;
          +import org.apache.rocketmq.filter.util.BloomFilterData;
          +
          +/**
          + * Filter data of consumer.
          + */
          +public class ConsumerFilterData {
          +
          + private String consumerGroup;
          + private String topic;
          + private String expression;
          + private String expressionType;
          + private transient Expression compiledExpression;
          + private long bornTime;
          + private long deadTime = 0;
          + private BloomFilterData bloomFilterData;
          + private long clientVersion;
          +
          + public boolean isDead()

          { + return this.deadTime >= this.bornTime; + }

          +
          + public long deadHowLong() {
          + if (isDead())

          { + return System.currentTimeMillis() - getDeadTime(); + }

          + return -1;
          + }
          +
          + /**
          + * Check this filter data has been used to calculate bit map when msg was stored in server.
          + *
          + * @param msgStoreTime
          + * @return
          + */
          + public boolean isMsgInLive(long msgStoreTime)

          { + return msgStoreTime > getBornTime(); + }

          +
          + public String getConsumerGroup()

          { + return consumerGroup; + }

          +
          + public void setConsumerGroup(final String consumerGroup)

          { + this.consumerGroup = consumerGroup; + }

          +
          + public String getTopic()

          { + return topic; + }

          +
          + public void setTopic(final String topic)

          { + this.topic = topic; + }

          +
          + public String getExpression()

          { + return expression; + }

          +
          + public void setExpression(final String expression)

          { + this.expression = expression; + }

          +
          + public String getExpressionType()

          { + return expressionType; + }

          +
          + public void setExpressionType(final String expressionType)

          { + this.expressionType = expressionType; + }

          +
          + public Expression getCompiledExpression()

          { + return compiledExpression; + }

          +
          + public void setCompiledExpression(final Expression compiledExpression)

          { + this.compiledExpression = compiledExpression; + }

          +
          + public long getBornTime()

          { + return bornTime; + }

          +
          + public void setBornTime(final long bornTime)

          { + this.bornTime = bornTime; + }

          +
          + public long getDeadTime()

          { + return deadTime; + }

          +
          + public void setDeadTime(final long deadTime)

          { + this.deadTime = deadTime; + }

          +
          + public BloomFilterData getBloomFilterData()

          { + return bloomFilterData; + }

          +
          + public void setBloomFilterData(final BloomFilterData bloomFilterData)

          { + this.bloomFilterData = bloomFilterData; + }

          +
          + public long getClientVersion()

          { + return clientVersion; + }

          +
          + public void setClientVersion(long clientVersion)

          { + this.clientVersion = clientVersion; + }

          +
          + @Override
          + public boolean equals(Object o) {
          — End diff –

          Could we use ToStringBuilder in Commons Lang3 to build equal and hash override method?

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108884189 — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/ConsumerFilterData.java — @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.broker.filter; + +import org.apache.rocketmq.filter.expression.Expression; +import org.apache.rocketmq.filter.util.BloomFilterData; + +/** + * Filter data of consumer. + */ +public class ConsumerFilterData { + + private String consumerGroup; + private String topic; + private String expression; + private String expressionType; + private transient Expression compiledExpression; + private long bornTime; + private long deadTime = 0; + private BloomFilterData bloomFilterData; + private long clientVersion; + + public boolean isDead() { + return this.deadTime >= this.bornTime; + } + + public long deadHowLong() { + if (isDead()) { + return System.currentTimeMillis() - getDeadTime(); + } + return -1; + } + + /** + * Check this filter data has been used to calculate bit map when msg was stored in server. + * + * @param msgStoreTime + * @return + */ + public boolean isMsgInLive(long msgStoreTime) { + return msgStoreTime > getBornTime(); + } + + public String getConsumerGroup() { + return consumerGroup; + } + + public void setConsumerGroup(final String consumerGroup) { + this.consumerGroup = consumerGroup; + } + + public String getTopic() { + return topic; + } + + public void setTopic(final String topic) { + this.topic = topic; + } + + public String getExpression() { + return expression; + } + + public void setExpression(final String expression) { + this.expression = expression; + } + + public String getExpressionType() { + return expressionType; + } + + public void setExpressionType(final String expressionType) { + this.expressionType = expressionType; + } + + public Expression getCompiledExpression() { + return compiledExpression; + } + + public void setCompiledExpression(final Expression compiledExpression) { + this.compiledExpression = compiledExpression; + } + + public long getBornTime() { + return bornTime; + } + + public void setBornTime(final long bornTime) { + this.bornTime = bornTime; + } + + public long getDeadTime() { + return deadTime; + } + + public void setDeadTime(final long deadTime) { + this.deadTime = deadTime; + } + + public BloomFilterData getBloomFilterData() { + return bloomFilterData; + } + + public void setBloomFilterData(final BloomFilterData bloomFilterData) { + this.bloomFilterData = bloomFilterData; + } + + public long getClientVersion() { + return clientVersion; + } + + public void setClientVersion(long clientVersion) { + this.clientVersion = clientVersion; + } + + @Override + public boolean equals(Object o) { — End diff – Could we use ToStringBuilder in Commons Lang3 to build equal and hash override method?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108886427

          — Diff: filter/src/main/java/org/apache/rocketmq/filter/parser/ParseException.java —
          @@ -0,0 +1,204 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +/* Generated By:JavaCC: Do not edit this line. ParseException.java Version 5.0 */
          — End diff –

          why java Version 5.0

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108886427 — Diff: filter/src/main/java/org/apache/rocketmq/filter/parser/ParseException.java — @@ -0,0 +1,204 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* Generated By:JavaCC: Do not edit this line. ParseException.java Version 5.0 */ — End diff – why java Version 5.0
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108887358

          — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/HashAlgorithm.java —
          @@ -0,0 +1,239 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.filter.util;
          +
          +import java.io.UnsupportedEncodingException;
          +import java.security.MessageDigest;
          +import java.security.NoSuchAlgorithmException;
          +import java.util.zip.CRC32;
          +
          +/**
          + * Note that all hash
          + * algorithms return 64-bits of hash, but only the lower 32-bits are
          + * significant. This allows a positive 32-bit number to be returned for all
          + * cases.
          + * <br>Don't change the order of algorithms, add new algorithm to last if you want.
          + */
          — End diff –

          Recommended commons codec's hashing algorithms

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108887358 — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/HashAlgorithm.java — @@ -0,0 +1,239 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.filter.util; + +import java.io.UnsupportedEncodingException; +import java.security.MessageDigest; +import java.security.NoSuchAlgorithmException; +import java.util.zip.CRC32; + +/** + * Note that all hash + * algorithms return 64-bits of hash, but only the lower 32-bits are + * significant. This allows a positive 32-bit number to be returned for all + * cases. + * <br>Don't change the order of algorithms, add new algorithm to last if you want. + */ — End diff – Recommended commons codec's hashing algorithms
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108886169

          — Diff: filter/src/main/java/org/apache/rocketmq/filter/expression/ConstantExpression.java —
          @@ -0,0 +1,156 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.filter.expression;
          +
          +/**
          + * Represents a constant expression
          + * <p>
          + * This class was taken from ActiveMQ org.apache.activemq.filter.ConstantExpression,
          + * but:
          — End diff –

          Great~

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108886169 — Diff: filter/src/main/java/org/apache/rocketmq/filter/expression/ConstantExpression.java — @@ -0,0 +1,156 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.filter.expression; + +/** + * Represents a constant expression + * <p> + * This class was taken from ActiveMQ org.apache.activemq.filter.ConstantExpression, + * but: — End diff – Great~
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108883414

          — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/CommitLogDispatcherCalcBitMap.java —
          @@ -0,0 +1,112 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.broker.filter;
          +
          +import org.apache.rocketmq.common.BrokerConfig;
          +import org.apache.rocketmq.common.constant.LoggerName;
          +import org.apache.rocketmq.filter.util.BitsArray;
          +import org.apache.rocketmq.store.CommitLogDispatcher;
          +import org.apache.rocketmq.store.DispatchRequest;
          +import org.slf4j.Logger;
          +import org.slf4j.LoggerFactory;
          +
          +import java.util.Collection;
          +import java.util.Iterator;
          +
          +/**
          + * Calculate bit map of filter.
          + */
          +public class CommitLogDispatcherCalcBitMap implements CommitLogDispatcher {
          +
          + private static final Logger log = LoggerFactory.getLogger(LoggerName.FILTER_LOGGER_NAME);
          +
          + protected final BrokerConfig brokerConfig;
          + protected final ConsumerFilterManager consumerFilterManager;
          +
          + public CommitLogDispatcherCalcBitMap(BrokerConfig brokerConfig, ConsumerFilterManager consumerFilterManager)

          { + this.brokerConfig = brokerConfig; + this.consumerFilterManager = consumerFilterManager; + }

          +
          + @Override
          + public void dispatch(DispatchRequest request) {
          + if (!this.brokerConfig.isEnableCalcFilterBitMap())

          { + return; + }

          +
          + try {
          +
          + Collection<ConsumerFilterData> filterDatas = consumerFilterManager.get(request.getTopic());
          +
          + if (filterDatas == null || filterDatas.isEmpty())

          { + return; + }

          +
          + Iterator<ConsumerFilterData> iterator = filterDatas.iterator();
          + BitsArray filterBitMap = BitsArray.create(
          + this.consumerFilterManager.getBloomFilter().getM()
          + );
          +
          + long startTime = System.currentTimeMillis();
          + while (iterator.hasNext()) {
          + ConsumerFilterData filterData = iterator.next();
          +
          + if (filterData.getCompiledExpression() == null) {
          + log.error("[BUG] Consumer in filter manager has no compiled expression! {}", filterData);
          + continue;
          + }
          +
          + if (filterData.getBloomFilterData() == null) {
          + log.error("[BUG] Consumer in filter manager has no bloom data! {}", filterData);
          + continue;
          + }
          +
          + Object ret = null;
          + try

          { + MessageEvaluationContext context = new MessageEvaluationContext(request.getPropertiesMap()); + + ret = filterData.getCompiledExpression().evaluate(context); + }

          catch (Throwable e) {
          + log.error("Calc filter bit map error!commitLogOffset=" + request.getCommitLogOffset() +
          — End diff –

          Please use {} replace +

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108883414 — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/CommitLogDispatcherCalcBitMap.java — @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.broker.filter; + +import org.apache.rocketmq.common.BrokerConfig; +import org.apache.rocketmq.common.constant.LoggerName; +import org.apache.rocketmq.filter.util.BitsArray; +import org.apache.rocketmq.store.CommitLogDispatcher; +import org.apache.rocketmq.store.DispatchRequest; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.Collection; +import java.util.Iterator; + +/** + * Calculate bit map of filter. + */ +public class CommitLogDispatcherCalcBitMap implements CommitLogDispatcher { + + private static final Logger log = LoggerFactory.getLogger(LoggerName.FILTER_LOGGER_NAME); + + protected final BrokerConfig brokerConfig; + protected final ConsumerFilterManager consumerFilterManager; + + public CommitLogDispatcherCalcBitMap(BrokerConfig brokerConfig, ConsumerFilterManager consumerFilterManager) { + this.brokerConfig = brokerConfig; + this.consumerFilterManager = consumerFilterManager; + } + + @Override + public void dispatch(DispatchRequest request) { + if (!this.brokerConfig.isEnableCalcFilterBitMap()) { + return; + } + + try { + + Collection<ConsumerFilterData> filterDatas = consumerFilterManager.get(request.getTopic()); + + if (filterDatas == null || filterDatas.isEmpty()) { + return; + } + + Iterator<ConsumerFilterData> iterator = filterDatas.iterator(); + BitsArray filterBitMap = BitsArray.create( + this.consumerFilterManager.getBloomFilter().getM() + ); + + long startTime = System.currentTimeMillis(); + while (iterator.hasNext()) { + ConsumerFilterData filterData = iterator.next(); + + if (filterData.getCompiledExpression() == null) { + log.error(" [BUG] Consumer in filter manager has no compiled expression! {}", filterData); + continue; + } + + if (filterData.getBloomFilterData() == null) { + log.error(" [BUG] Consumer in filter manager has no bloom data! {}", filterData); + continue; + } + + Object ret = null; + try { + MessageEvaluationContext context = new MessageEvaluationContext(request.getPropertiesMap()); + + ret = filterData.getCompiledExpression().evaluate(context); + } catch (Throwable e) { + log.error("Calc filter bit map error!commitLogOffset=" + request.getCommitLogOffset() + — End diff – Please use {} replace +
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108887571

          — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/BloomFilter.java —
          @@ -0,0 +1,332 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.filter.util;
          +
          +import java.util.Arrays;
          +
          +/**
          + * Simple implement of bloom filter.
          + */
          +public class BloomFilter {
          — End diff –

          Recommended existent mature BloomFilter class

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108887571 — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/BloomFilter.java — @@ -0,0 +1,332 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.filter.util; + +import java.util.Arrays; + +/** + * Simple implement of bloom filter. + */ +public class BloomFilter { — End diff – Recommended existent mature BloomFilter class
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108884747

          — Diff: broker/src/test/java/org/apache/rocketmq/broker/filter/CommitLogDispatcherCalcBitMapTest.java —
          @@ -0,0 +1,213 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.broker.filter;
          +
          +import org.apache.rocketmq.common.BrokerConfig;
          +import org.apache.rocketmq.common.filter.ExpressionType;
          +import org.apache.rocketmq.filter.util.BitsArray;
          +import org.apache.rocketmq.store.DispatchRequest;
          +import org.junit.Test;
          +
          +import java.util.Collection;
          +import java.util.HashMap;
          +import java.util.Map;
          +import java.util.UUID;
          +
          +import static org.assertj.core.api.Assertions.assertThat;
          +
          +public class CommitLogDispatcherCalcBitMapTest {
          +
          + @Test
          + public void testDispatch_filterDataIllegal() {
          + BrokerConfig brokerConfig = new BrokerConfig();
          + brokerConfig.setEnableCalcFilterBitMap(true);
          +
          + ConsumerFilterManager filterManager = new ConsumerFilterManager();
          +
          + try

          { + Thread.sleep(1000); + }

          catch (InterruptedException e) {
          + e.printStackTrace();
          — End diff –

          please remove it

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108884747 — Diff: broker/src/test/java/org/apache/rocketmq/broker/filter/CommitLogDispatcherCalcBitMapTest.java — @@ -0,0 +1,213 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.broker.filter; + +import org.apache.rocketmq.common.BrokerConfig; +import org.apache.rocketmq.common.filter.ExpressionType; +import org.apache.rocketmq.filter.util.BitsArray; +import org.apache.rocketmq.store.DispatchRequest; +import org.junit.Test; + +import java.util.Collection; +import java.util.HashMap; +import java.util.Map; +import java.util.UUID; + +import static org.assertj.core.api.Assertions.assertThat; + +public class CommitLogDispatcherCalcBitMapTest { + + @Test + public void testDispatch_filterDataIllegal() { + BrokerConfig brokerConfig = new BrokerConfig(); + brokerConfig.setEnableCalcFilterBitMap(true); + + ConsumerFilterManager filterManager = new ConsumerFilterManager(); + + try { + Thread.sleep(1000); + } catch (InterruptedException e) { + e.printStackTrace(); — End diff – please remove it
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108887819

          — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/LRUCache.java —
          @@ -0,0 +1,92 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.filter.util;
          +
          +import java.util.LinkedHashMap;
          +import java.util.Map;
          +
          +/**
          + * This class was taken from ActiveMQ org.apache.activemq.util.LRUCache,
          + * @param <K>
          + * @param <V>
          + */
          +public class LRUCache<K, V> extends LinkedHashMap<K, V> {
          — End diff –

          Can we remove this class

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108887819 — Diff: filter/src/main/java/org/apache/rocketmq/filter/util/LRUCache.java — @@ -0,0 +1,92 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.filter.util; + +import java.util.LinkedHashMap; +import java.util.Map; + +/** + * This class was taken from ActiveMQ org.apache.activemq.util.LRUCache, + * @param <K> + * @param <V> + */ +public class LRUCache<K, V> extends LinkedHashMap<K, V> { — End diff – Can we remove this class
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108885505

          — Diff: conf/logback_broker.xml —
          @@ -222,6 +222,29 @@
          <appender-ref ref="RocketmqRebalanceLockAppender_inner"/>
          </appender>

          + <appender name="RocketmqFilterAppender_inner"
          + class="ch.qos.logback.core.rolling.RollingFileAppender">
          + <file>$

          {user.home}/logs/rocketmqlogs/filter.log</file>
          + <append>true</append>
          + <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy">
          + <fileNamePattern>${user.home}

          /logs/rocketmqlogs/otherdays/filter.%i.log
          — End diff –

          otherdays?

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108885505 — Diff: conf/logback_broker.xml — @@ -222,6 +222,29 @@ <appender-ref ref="RocketmqRebalanceLockAppender_inner"/> </appender> + <appender name="RocketmqFilterAppender_inner" + class="ch.qos.logback.core.rolling.RollingFileAppender"> + <file>$ {user.home}/logs/rocketmqlogs/filter.log</file> + <append>true</append> + <rollingPolicy class="ch.qos.logback.core.rolling.FixedWindowRollingPolicy"> + <fileNamePattern>${user.home} /logs/rocketmqlogs/otherdays/filter.%i.log — End diff – otherdays?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108883657

          — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/CommitLogDispatcherCalcBitMap.java —
          @@ -0,0 +1,112 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.broker.filter;
          +
          +import org.apache.rocketmq.common.BrokerConfig;
          +import org.apache.rocketmq.common.constant.LoggerName;
          +import org.apache.rocketmq.filter.util.BitsArray;
          +import org.apache.rocketmq.store.CommitLogDispatcher;
          +import org.apache.rocketmq.store.DispatchRequest;
          +import org.slf4j.Logger;
          +import org.slf4j.LoggerFactory;
          +
          +import java.util.Collection;
          +import java.util.Iterator;
          +
          +/**
          + * Calculate bit map of filter.
          + */
          +public class CommitLogDispatcherCalcBitMap implements CommitLogDispatcher {
          +
          + private static final Logger log = LoggerFactory.getLogger(LoggerName.FILTER_LOGGER_NAME);
          +
          + protected final BrokerConfig brokerConfig;
          + protected final ConsumerFilterManager consumerFilterManager;
          +
          + public CommitLogDispatcherCalcBitMap(BrokerConfig brokerConfig, ConsumerFilterManager consumerFilterManager)

          { + this.brokerConfig = brokerConfig; + this.consumerFilterManager = consumerFilterManager; + }

          +
          + @Override
          + public void dispatch(DispatchRequest request) {
          + if (!this.brokerConfig.isEnableCalcFilterBitMap())

          { + return; + }

          +
          + try {
          +
          + Collection<ConsumerFilterData> filterDatas = consumerFilterManager.get(request.getTopic());
          +
          + if (filterDatas == null || filterDatas.isEmpty())

          { + return; + }

          +
          + Iterator<ConsumerFilterData> iterator = filterDatas.iterator();
          + BitsArray filterBitMap = BitsArray.create(
          + this.consumerFilterManager.getBloomFilter().getM()
          + );
          +
          + long startTime = System.currentTimeMillis();
          + while (iterator.hasNext()) {
          + ConsumerFilterData filterData = iterator.next();
          +
          + if (filterData.getCompiledExpression() == null) {
          + log.error("[BUG] Consumer in filter manager has no compiled expression! {}", filterData);
          + continue;
          + }
          +
          + if (filterData.getBloomFilterData() == null) {
          + log.error("[BUG] Consumer in filter manager has no bloom data! {}", filterData);
          + continue;
          + }
          +
          + Object ret = null;
          + try

          { + MessageEvaluationContext context = new MessageEvaluationContext(request.getPropertiesMap()); + + ret = filterData.getCompiledExpression().evaluate(context); + }

          catch (Throwable e)

          { + log.error("Calc filter bit map error!commitLogOffset=" + request.getCommitLogOffset() + + ", consumer=" + filterData, e); + }

          +
          + log.debug("Result of Calc bit map:ret={}, data={}, props={}, offset={}", ret, filterData, request.getPropertiesMap(), request.getCommitLogOffset());
          +
          + // eval true
          + if (ret != null && ret instanceof Boolean && (Boolean) ret)

          { + consumerFilterManager.getBloomFilter().hashTo( + filterData.getBloomFilterData(), + filterBitMap + ); + }

          + }
          +
          + request.setBitMap(filterBitMap.bytes());
          +
          + long eclipseTime = System.currentTimeMillis() - startTime;
          + // 1ms
          + if (eclipseTime >= 1) {
          + log.warn("Spend {} ms to calc bit map, consumerNum={}, topic={}", eclipseTime, filterDatas.size(), request.getTopic());
          + }
          + } catch (Throwable e) {
          + log.error("Calc bit map error! topic=" + request.getTopic() + ", offset=" + request.getCommitLogOffset()
          — End diff –

          Follow the previous comment

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108883657 — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/CommitLogDispatcherCalcBitMap.java — @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.broker.filter; + +import org.apache.rocketmq.common.BrokerConfig; +import org.apache.rocketmq.common.constant.LoggerName; +import org.apache.rocketmq.filter.util.BitsArray; +import org.apache.rocketmq.store.CommitLogDispatcher; +import org.apache.rocketmq.store.DispatchRequest; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import java.util.Collection; +import java.util.Iterator; + +/** + * Calculate bit map of filter. + */ +public class CommitLogDispatcherCalcBitMap implements CommitLogDispatcher { + + private static final Logger log = LoggerFactory.getLogger(LoggerName.FILTER_LOGGER_NAME); + + protected final BrokerConfig brokerConfig; + protected final ConsumerFilterManager consumerFilterManager; + + public CommitLogDispatcherCalcBitMap(BrokerConfig brokerConfig, ConsumerFilterManager consumerFilterManager) { + this.brokerConfig = brokerConfig; + this.consumerFilterManager = consumerFilterManager; + } + + @Override + public void dispatch(DispatchRequest request) { + if (!this.brokerConfig.isEnableCalcFilterBitMap()) { + return; + } + + try { + + Collection<ConsumerFilterData> filterDatas = consumerFilterManager.get(request.getTopic()); + + if (filterDatas == null || filterDatas.isEmpty()) { + return; + } + + Iterator<ConsumerFilterData> iterator = filterDatas.iterator(); + BitsArray filterBitMap = BitsArray.create( + this.consumerFilterManager.getBloomFilter().getM() + ); + + long startTime = System.currentTimeMillis(); + while (iterator.hasNext()) { + ConsumerFilterData filterData = iterator.next(); + + if (filterData.getCompiledExpression() == null) { + log.error(" [BUG] Consumer in filter manager has no compiled expression! {}", filterData); + continue; + } + + if (filterData.getBloomFilterData() == null) { + log.error(" [BUG] Consumer in filter manager has no bloom data! {}", filterData); + continue; + } + + Object ret = null; + try { + MessageEvaluationContext context = new MessageEvaluationContext(request.getPropertiesMap()); + + ret = filterData.getCompiledExpression().evaluate(context); + } catch (Throwable e) { + log.error("Calc filter bit map error!commitLogOffset=" + request.getCommitLogOffset() + + ", consumer=" + filterData, e); + } + + log.debug("Result of Calc bit map:ret={}, data={}, props={}, offset={}", ret, filterData, request.getPropertiesMap(), request.getCommitLogOffset()); + + // eval true + if (ret != null && ret instanceof Boolean && (Boolean) ret) { + consumerFilterManager.getBloomFilter().hashTo( + filterData.getBloomFilterData(), + filterBitMap + ); + } + } + + request.setBitMap(filterBitMap.bytes()); + + long eclipseTime = System.currentTimeMillis() - startTime; + // 1ms + if (eclipseTime >= 1) { + log.warn("Spend {} ms to calc bit map, consumerNum={}, topic={}", eclipseTime, filterDatas.size(), request.getTopic()); + } + } catch (Throwable e) { + log.error("Calc bit map error! topic=" + request.getTopic() + ", offset=" + request.getCommitLogOffset() — End diff – Follow the previous comment
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108886049

          — Diff: filter/src/main/java/org/apache/rocketmq/filter/expression/BinaryExpression.java —
          @@ -0,0 +1,91 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.filter.expression;
          +
          +/**
          + * An expression which performs an operation on two expression values.
          + * <p>
          + * This class was taken from ActiveMQ org.apache.activemq.filter.BinaryExpression,
          — End diff –

          Cool~

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108886049 — Diff: filter/src/main/java/org/apache/rocketmq/filter/expression/BinaryExpression.java — @@ -0,0 +1,91 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.filter.expression; + +/** + * An expression which performs an operation on two expression values. + * <p> + * This class was taken from ActiveMQ org.apache.activemq.filter.BinaryExpression, — End diff – Cool~
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108883110

          — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/CommitLogDispatcherCalcBitMap.java —
          @@ -0,0 +1,112 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.broker.filter;
          +
          +import org.apache.rocketmq.common.BrokerConfig;
          +import org.apache.rocketmq.common.constant.LoggerName;
          +import org.apache.rocketmq.filter.util.BitsArray;
          — End diff –

          BitMap ? Could we use the roaring bitmap, https://github.com/RoaringBitmap/RoaringBitmap

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108883110 — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/CommitLogDispatcherCalcBitMap.java — @@ -0,0 +1,112 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.broker.filter; + +import org.apache.rocketmq.common.BrokerConfig; +import org.apache.rocketmq.common.constant.LoggerName; +import org.apache.rocketmq.filter.util.BitsArray; — End diff – BitMap ? Could we use the roaring bitmap, https://github.com/RoaringBitmap/RoaringBitmap
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108883867

          — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/ConsumerFilterData.java —
          @@ -0,0 +1,180 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.rocketmq.broker.filter;
          +
          +import org.apache.rocketmq.filter.expression.Expression;
          +import org.apache.rocketmq.filter.util.BloomFilterData;
          +
          +/**
          + * Filter data of consumer.
          + */
          +public class ConsumerFilterData {
          +
          + private String consumerGroup;
          + private String topic;
          + private String expression;
          + private String expressionType;
          + private transient Expression compiledExpression;
          + private long bornTime;
          + private long deadTime = 0;
          + private BloomFilterData bloomFilterData;
          + private long clientVersion;
          +
          + public boolean isDead()

          { + return this.deadTime >= this.bornTime; + }

          +
          + public long deadHowLong() {
          — End diff –

          what is deadhowlong

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108883867 — Diff: broker/src/main/java/org/apache/rocketmq/broker/filter/ConsumerFilterData.java — @@ -0,0 +1,180 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.rocketmq.broker.filter; + +import org.apache.rocketmq.filter.expression.Expression; +import org.apache.rocketmq.filter.util.BloomFilterData; + +/** + * Filter data of consumer. + */ +public class ConsumerFilterData { + + private String consumerGroup; + private String topic; + private String expression; + private String expressionType; + private transient Expression compiledExpression; + private long bornTime; + private long deadTime = 0; + private BloomFilterData bloomFilterData; + private long clientVersion; + + public boolean isDead() { + return this.deadTime >= this.bornTime; + } + + public long deadHowLong() { — End diff – what is deadhowlong
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/10844981/badge)](https://coveralls.io/builds/10844981)

          Coverage increased (+3.7%) to 35.609% when pulling *67fe978f1a35c8c8ebdd1e70beb4944a4043c77a on vsair:ROCKETMQ-121* into *45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/10844981/badge)](https://coveralls.io/builds/10844981 ) Coverage increased (+3.7%) to 35.609% when pulling * 67fe978f1a35c8c8ebdd1e70beb4944a4043c77a on vsair: ROCKETMQ-121 * into * 45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/10844981/badge)](https://coveralls.io/builds/10844981)

          Coverage increased (+3.7%) to 35.609% when pulling *67fe978f1a35c8c8ebdd1e70beb4944a4043c77a on vsair:ROCKETMQ-121* into *45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/10844981/badge)](https://coveralls.io/builds/10844981 ) Coverage increased (+3.7%) to 35.609% when pulling * 67fe978f1a35c8c8ebdd1e70beb4944a4043c77a on vsair: ROCKETMQ-121 * into * 45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/10844981/badge)](https://coveralls.io/builds/10844981)

          Coverage increased (+3.7%) to 35.609% when pulling *67fe978f1a35c8c8ebdd1e70beb4944a4043c77a on vsair:ROCKETMQ-121* into *45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/10844981/badge)](https://coveralls.io/builds/10844981 ) Coverage increased (+3.7%) to 35.609% when pulling * 67fe978f1a35c8c8ebdd1e70beb4944a4043c77a on vsair: ROCKETMQ-121 * into * 45a64fd6a8ab66a8b0eb30cbe2dffd59b19274f9 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on a diff in the pull request:

          https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108881956

          — Diff: broker/src/main/java/org/apache/rocketmq/broker/client/ConsumerIdsChangeListener.java —
          @@ -16,9 +16,7 @@
          */
          package org.apache.rocketmq.broker.client;

          -import io.netty.channel.Channel;
          -import java.util.List;
          -
          public interface ConsumerIdsChangeListener {

          • void consumerIdsChanged(final String group, final List<Channel> channels);
            +
              • End diff –

          Great refactoring

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on a diff in the pull request: https://github.com/apache/incubator-rocketmq/pull/82#discussion_r108881956 — Diff: broker/src/main/java/org/apache/rocketmq/broker/client/ConsumerIdsChangeListener.java — @@ -16,9 +16,7 @@ */ package org.apache.rocketmq.broker.client; -import io.netty.channel.Channel; -import java.util.List; - public interface ConsumerIdsChangeListener { void consumerIdsChanged(final String group, final List<Channel> channels); + End diff – Great refactoring
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user vongosling commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          @shroman @lizhanhui How do you think about this feature~

          Show
          githubbot ASF GitHub Bot added a comment - Github user vongosling commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 @shroman @lizhanhui How do you think about this feature~
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/10730651/badge)](https://coveralls.io/builds/10730651)

          Coverage increased (+3.8%) to 35.617% when pulling *a9cea30982d30ce7cc5b1842eb86e87fecc6e682 on vsair:ROCKETMQ-121* into *72e6def1441083c6c5a8c4b831eebefcbb618b56 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/10730651/badge)](https://coveralls.io/builds/10730651 ) Coverage increased (+3.8%) to 35.617% when pulling * a9cea30982d30ce7cc5b1842eb86e87fecc6e682 on vsair: ROCKETMQ-121 * into * 72e6def1441083c6c5a8c4b831eebefcbb618b56 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/10730651/badge)](https://coveralls.io/builds/10730651)

          Coverage increased (+3.8%) to 35.617% when pulling *a9cea30982d30ce7cc5b1842eb86e87fecc6e682 on vsair:ROCKETMQ-121* into *72e6def1441083c6c5a8c4b831eebefcbb618b56 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/10730651/badge)](https://coveralls.io/builds/10730651 ) Coverage increased (+3.8%) to 35.617% when pulling * a9cea30982d30ce7cc5b1842eb86e87fecc6e682 on vsair: ROCKETMQ-121 * into * 72e6def1441083c6c5a8c4b831eebefcbb618b56 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/10730651/badge)](https://coveralls.io/builds/10730651)

          Coverage increased (+3.8%) to 35.617% when pulling *a9cea30982d30ce7cc5b1842eb86e87fecc6e682 on vsair:ROCKETMQ-121* into *72e6def1441083c6c5a8c4b831eebefcbb618b56 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/10730651/badge)](https://coveralls.io/builds/10730651 ) Coverage increased (+3.8%) to 35.617% when pulling * a9cea30982d30ce7cc5b1842eb86e87fecc6e682 on vsair: ROCKETMQ-121 * into * 72e6def1441083c6c5a8c4b831eebefcbb618b56 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/10714177/badge)](https://coveralls.io/builds/10714177)

          Coverage increased (+3.4%) to 35.261% when pulling *e258309ed35d71f74e21823dcb8ce912cb7682eb on vsair:ROCKETMQ-121* into *72e6def1441083c6c5a8c4b831eebefcbb618b56 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/10714177/badge)](https://coveralls.io/builds/10714177 ) Coverage increased (+3.4%) to 35.261% when pulling * e258309ed35d71f74e21823dcb8ce912cb7682eb on vsair: ROCKETMQ-121 * into * 72e6def1441083c6c5a8c4b831eebefcbb618b56 on apache:develop *.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user coveralls commented on the issue:

          https://github.com/apache/incubator-rocketmq/pull/82

          [![Coverage Status](https://coveralls.io/builds/10714177/badge)](https://coveralls.io/builds/10714177)

          Coverage increased (+3.4%) to 35.261% when pulling *e258309ed35d71f74e21823dcb8ce912cb7682eb on vsair:ROCKETMQ-121* into *72e6def1441083c6c5a8c4b831eebefcbb618b56 on apache:develop*.

          Show
          githubbot ASF GitHub Bot added a comment - Github user coveralls commented on the issue: https://github.com/apache/incubator-rocketmq/pull/82 [! [Coverage Status] ( https://coveralls.io/builds/10714177/badge)](https://coveralls.io/builds/10714177 ) Coverage increased (+3.4%) to 35.261% when pulling * e258309ed35d71f74e21823dcb8ce912cb7682eb on vsair: ROCKETMQ-121 * into * 72e6def1441083c6c5a8c4b831eebefcbb618b56 on apache:develop *.
          Hide
          zhangke Rich Zhang added a comment -

          Cool.

          Eric, could you initiate a discussion in dev mail list? As far as I know, ActiveMQ also use JavaCC to implement this functionality.

          Show
          zhangke Rich Zhang added a comment - Cool. Eric, could you initiate a discussion in dev mail list? As far as I know, ActiveMQ also use JavaCC to implement this functionality.
          Hide
          Yukon yukon added a comment -

          Hi,

          Since you already have a implementation, could you please send a PR to rocketmq with detailed descriptions, design, and so on.
          And then let's discuss it at the dev mailing list, maybe you two guys can work together to finish this feature.

          Show
          Yukon yukon added a comment - Hi, Since you already have a implementation, could you please send a PR to rocketmq with detailed descriptions, design, and so on. And then let's discuss it at the dev mailing list, maybe you two guys can work together to finish this feature.
          Hide
          eric.liu Eric Liu added a comment -

          I already had an edition that filter according to message properties.

          My edition support :
          1. numeric comparison
          2. null check
          3. etc...

          Also the edition could be extended easily to have more features because it is implemented by javaCC.

          Besides, consider the pull pattern of RocketMQ, the edition also support the scenario of high subscription ratio, by calculating the result advanced.

          See pr for more detail later.

          So, could I be the assignee of this issue?

          Show
          eric.liu Eric Liu added a comment - I already had an edition that filter according to message properties. My edition support : 1. numeric comparison 2. null check 3. etc... Also the edition could be extended easily to have more features because it is implemented by javaCC. Besides, consider the pull pattern of RocketMQ, the edition also support the scenario of high subscription ratio, by calculating the result advanced. See pr for more detail later. So, could I be the assignee of this issue?

            People

            • Assignee:
              eric.liu Eric Liu
              Reporter:
              Yukon yukon
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development