Details

    • Type: Task Task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.0
    • Component/s: contrib
    • Labels:
      None

      Description

      The wikisearch example is a submodule of accumulo examples, but it is complex enough to be a separate contrib. Its sibling submodule, examples-simple, could then be moved back to an accumulo submodule.

        Activity

        Billie Rinaldi created issue -
        Gavin made changes -
        Field Original Value New Value
        Workflow no-reopen-closed, patch-avail [ 12668814 ] patch-available, re-open possible [ 12671828 ]
        Hide
        Dave Marion added a comment -

        Agree that its complex. Have you considered moving it out of the build entirely so that you don't have to keep it up to date. By moving it out (to GitHub as an example) it could be maintained on its own schedule.

        Show
        Dave Marion added a comment - Agree that its complex. Have you considered moving it out of the build entirely so that you don't have to keep it up to date. By moving it out (to GitHub as an example) it could be maintained on its own schedule.
        Hide
        Adam Fuchs added a comment -

        We have actually had several IRL discussions on this matter. I think that it is central enough in showing off the core features that we want to maintain it as part of the offical Accumulo project. At the same time, we should try reduce the complexity of the core project to speed up development there. I think the separate contrib directory is a good compromise.

        Show
        Adam Fuchs added a comment - We have actually had several IRL discussions on this matter. I think that it is central enough in showing off the core features that we want to maintain it as part of the offical Accumulo project. At the same time, we should try reduce the complexity of the core project to speed up development there. I think the separate contrib directory is a good compromise.
        Hide
        Chris Waring added a comment -

        I would agree. This would allow more flexibility.

        Show
        Chris Waring added a comment - I would agree. This would allow more flexibility.
        Hide
        William Slacum added a comment -

        +1

        Show
        William Slacum added a comment - +1
        Hide
        Chris Waring added a comment -

        So as to not have ambiguity forced on me by order of submission. I concur that is should be move out. Especially given most of the developers of that example are not listed as contributor or committers to Accumulo.

        Show
        Chris Waring added a comment - So as to not have ambiguity forced on me by order of submission. I concur that is should be move out. Especially given most of the developers of that example are not listed as contributor or committers to Accumulo.
        Hide
        Dave Marion added a comment -

        Another reason to move it out would be to allow inclusion of dependencies that are not allowed by ASF.

        Show
        Dave Marion added a comment - Another reason to move it out would be to allow inclusion of dependencies that are not allowed by ASF.
        Hide
        Adam Fuchs added a comment -

        That would also be a good reason to keep it in.

        Show
        Adam Fuchs added a comment - That would also be a good reason to keep it in.
        Hide
        Eric Newton added a comment -

        -1

        I would like to keep it in. if we move it out, we don't test it, we don't keep it up-to-date. I agree that it should be its own sub-project, eventually.

        It is a unique feature. Nothing else works like this, except maybe Solr, and to my knowledge, it isn't as extensible. Cell level security is very interesting to some people, high ingest rates to even more, but custom, distributed, index-based joins is interesting to a much larger user base.

        Show
        Eric Newton added a comment - -1 I would like to keep it in. if we move it out, we don't test it, we don't keep it up-to-date. I agree that it should be its own sub-project, eventually. It is a unique feature. Nothing else works like this, except maybe Solr, and to my knowledge, it isn't as extensible. Cell level security is very interesting to some people, high ingest rates to even more, but custom, distributed, index-based joins is interesting to a much larger user base.
        Hide
        Josh Elser added a comment -

        0 (or consider this a non-vote at this point)

        The wikisearch example is a very difficult project to determine ownership over. The knowledge of the implementation and the (subtle) caveats is primarily outside the knowledge of the current list of contributors, like Chris stated. Meaning, it doesn't make sense to keep it underneath the Accumulo trunk as those who fix it have to operate proxied through other committers. The wikisearch code already suffers from rot, due to less direct love than it deserves/needs. Saying that we need to reduce complexity of the code doesn't mean anything if no one can actually work on improving things.

        Although, I agree with Eric that the wikisearch code is an excellent example of the features, performance, and application of Accumulo which would make sense to keep with Accumulo to ensure that a real-life application is tied with each Accumulo release. For those who want to do something "serious" with Accumulo, the wikisearch can (and I would argue, should) be an excellent place to begin to learn about the intricacies and best-practices. There's far too much contained in functioning code that documentation can never encompass.

        Something needs to change to keep the wikisearch code at the level it should be, but I'm undecided if the medium of hosting will really matter in the long run. Ill-maintained code will suffer no matter where it's kept, but change needs to happen before the example is moot.

        Show
        Josh Elser added a comment - 0 (or consider this a non-vote at this point) The wikisearch example is a very difficult project to determine ownership over. The knowledge of the implementation and the (subtle) caveats is primarily outside the knowledge of the current list of contributors, like Chris stated. Meaning, it doesn't make sense to keep it underneath the Accumulo trunk as those who fix it have to operate proxied through other committers. The wikisearch code already suffers from rot, due to less direct love than it deserves/needs. Saying that we need to reduce complexity of the code doesn't mean anything if no one can actually work on improving things. Although, I agree with Eric that the wikisearch code is an excellent example of the features, performance, and application of Accumulo which would make sense to keep with Accumulo to ensure that a real-life application is tied with each Accumulo release. For those who want to do something "serious" with Accumulo, the wikisearch can (and I would argue, should) be an excellent place to begin to learn about the intricacies and best-practices. There's far too much contained in functioning code that documentation can never encompass. Something needs to change to keep the wikisearch code at the level it should be, but I'm undecided if the medium of hosting will really matter in the long run. Ill-maintained code will suffer no matter where it's kept, but change needs to happen before the example is moot.
        Hide
        Billie Rinaldi added a comment -

        I'm neutral on whether we should move it to a contrib or to GitHub, as long as we don't leave it where it is. We should keep in mind that we haven't figured out how we want to release contribs yet.

        Show
        Billie Rinaldi added a comment - I'm neutral on whether we should move it to a contrib or to GitHub, as long as we don't leave it where it is. We should keep in mind that we haven't figured out how we want to release contribs yet.
        Hide
        David Medinets added a comment -

        +1 for the github option. The wikisearch code, to me, is more of a prototype application than a part of Accumulo. If it was a github project, it could have a different set of contributors?

        Show
        David Medinets added a comment - +1 for the github option. The wikisearch code, to me, is more of a prototype application than a part of Accumulo. If it was a github project, it could have a different set of contributors?
        Hide
        Keith Turner added a comment -

        I think it makes sense to move it out. It should be able to evolve at its own pace, independent of the Accumulo release schedule. I think its also nice to have newer version of wikisearch work with older versions of Accumulo. For example if new features are added to wikisearch and its released again, it would be nice if the newer version worked with Accumulo 1.4.X. If its versioned with Accumulo there is a temptation to take advantage of newer feature of Accumulo that do not exist in older versions.

        If its in contrib, it can be built and placed in a contrib dir at release time. Also, it can evolve at its own pace. So when we release 1.5.0, we can include wikisearch.

        If its in apache svn or on github, it would be nice if stayed compatible with the apache license so it can be included w/ an Accumulo release.

        Show
        Keith Turner added a comment - I think it makes sense to move it out. It should be able to evolve at its own pace, independent of the Accumulo release schedule. I think its also nice to have newer version of wikisearch work with older versions of Accumulo. For example if new features are added to wikisearch and its released again, it would be nice if the newer version worked with Accumulo 1.4.X. If its versioned with Accumulo there is a temptation to take advantage of newer feature of Accumulo that do not exist in older versions. If its in contrib, it can be built and placed in a contrib dir at release time. Also, it can evolve at its own pace. So when we release 1.5.0, we can include wikisearch. If its in apache svn or on github, it would be nice if stayed compatible with the apache license so it can be included w/ an Accumulo release.
        Hide
        Adam Fuchs added a comment -

        Does anyone know if moving WikiSearch to Github would preclude us from including it as part of future releases? I know there's ASF policy saying that code for ASF projects must be kept in ASF's SVN repository – does that extend to contrib parts of a release?

        Show
        Adam Fuchs added a comment - Does anyone know if moving WikiSearch to Github would preclude us from including it as part of future releases? I know there's ASF policy saying that code for ASF projects must be kept in ASF's SVN repository – does that extend to contrib parts of a release?
        Hide
        Keith Turner added a comment -

        We currently include 3rd party jars in our release, as long as the license is compatible. I would think the same would apply to contrib. We are not saying whats in contrib is Apache Accumulo code, so it probably does not need to be in Aapche svn.

        Show
        Keith Turner added a comment - We currently include 3rd party jars in our release, as long as the license is compatible. I would think the same would apply to contrib. We are not saying whats in contrib is Apache Accumulo code, so it probably does not need to be in Aapche svn.
        Hide
        Chris A. Mattmann added a comment -

        Hi Adam, and Keith: any reason to move it to Github if it's currently within the ASF Accumulo project? I guess I'm not seeing the logic about removing it in general? Won't that splinter the development?

        Show
        Chris A. Mattmann added a comment - Hi Adam, and Keith: any reason to move it to Github if it's currently within the ASF Accumulo project? I guess I'm not seeing the logic about removing it in general? Won't that splinter the development?
        Hide
        Keith Turner added a comment -

        The original proposal was to move it from

        https://svn.apache.org/repos/asf/accumulo/trunk

        to

        https://svn.apache.org/repos/asf/accumulo/contrib

        Then github entered the conversation as another place it could move to. I do not have a strong opinion about where it should go, but I would like to see it move from trunk to contrib or github.

        Show
        Keith Turner added a comment - The original proposal was to move it from https://svn.apache.org/repos/asf/accumulo/trunk to https://svn.apache.org/repos/asf/accumulo/contrib Then github entered the conversation as another place it could move to. I do not have a strong opinion about where it should go, but I would like to see it move from trunk to contrib or github.
        Hide
        Chris A. Mattmann added a comment -

        I realize I'm not on the PMC or anything, so take it with a grain of salt (though as an ASF member, and fellow govvie, I care about the project a lot! )

        +1 on moving things, it looks like you are trying to get it out of the realm of being baked into the trunk, and into its own release cycle.

        As for deciding to move something to Github, when it's originally ASF code, why bother?

        Show
        Chris A. Mattmann added a comment - I realize I'm not on the PMC or anything, so take it with a grain of salt (though as an ASF member, and fellow govvie, I care about the project a lot! ) +1 on moving things, it looks like you are trying to get it out of the realm of being baked into the trunk, and into its own release cycle. As for deciding to move something to Github, when it's originally ASF code, why bother?
        Hide
        Chris A. Mattmann added a comment -

        I just noticed this comment:

        We are not saying whats in contrib is Apache Accumulo code

        I don't get it – code that's in the Apache Accumulo SVN and that's part of the bits there is Apache Accumulo code – whether it's released as part of the (set of) product(s) that the Apache Accumulo project releases is up for discussion, but if the code was developed in Apache the provenance is that it's Apache code.

        Show
        Chris A. Mattmann added a comment - I just noticed this comment: We are not saying whats in contrib is Apache Accumulo code I don't get it – code that's in the Apache Accumulo SVN and that's part of the bits there is Apache Accumulo code – whether it's released as part of the (set of) product(s) that the Apache Accumulo project releases is up for discussion, but if the code was developed in Apache the provenance is that it's Apache code.
        Hide
        Keith Turner added a comment -

        Chris M,

        You are correct, from a provenance stand point its Apache Accumulo code. It has went throught all of the required legal steps. I was thinking of it from a technical standpoint. The wikisearch code is not required to run Accumulo. Its a project the builds on top of Accumulo.

        Show
        Keith Turner added a comment - Chris M, You are correct, from a provenance stand point its Apache Accumulo code. It has went throught all of the required legal steps. I was thinking of it from a technical standpoint. The wikisearch code is not required to run Accumulo. Its a project the builds on top of Accumulo.
        Hide
        Josh Elser added a comment -

        I think the biggest issue has been that most of us who have been a part of the wikisearch development are not currently Accumulo developers (in other words, without svn commit access).

        Correct me if I'm wrong, anyone, but I believe Github was suggested as an alternative to the ASF SVN if it was decided that those who will be maintaing the wikisearch example will stay disjoint from those who are Accumulo committers.

        Show
        Josh Elser added a comment - I think the biggest issue has been that most of us who have been a part of the wikisearch development are not currently Accumulo developers (in other words, without svn commit access). Correct me if I'm wrong, anyone, but I believe Github was suggested as an alternative to the ASF SVN if it was decided that those who will be maintaing the wikisearch example will stay disjoint from those who are Accumulo committers.
        Hide
        Chris A. Mattmann added a comment -

        Hi Josh:

        I think the biggest issue has been that most of us who have been a part of the wikisearch development are not currently Accumulo developers (in other words, without svn commit access).

        Sorry to hear that – how much contribution would you say you've been doing?

        Correct me if I'm wrong, anyone, but I believe Github was suggested as an alternative to the ASF SVN if it was decided that those who will be maintaing the wikisearch example will stay disjoint from those who are Accumulo committers.

        Do you have any context on this? Like an email thread on list where this was discussed?

        Show
        Chris A. Mattmann added a comment - Hi Josh: I think the biggest issue has been that most of us who have been a part of the wikisearch development are not currently Accumulo developers (in other words, without svn commit access). Sorry to hear that – how much contribution would you say you've been doing? Correct me if I'm wrong, anyone, but I believe Github was suggested as an alternative to the ASF SVN if it was decided that those who will be maintaing the wikisearch example will stay disjoint from those who are Accumulo committers. Do you have any context on this? Like an email thread on list where this was discussed?
        Hide
        Josh Elser added a comment -

        Sorry to hear that – how much contribution would you say you've been doing?

        The few patches I've made have been coordinated with Eric Newton. Some on jira, some off.

        Do you have any context on this? Like an email thread on list where this was discussed?

        No, most of this was discussed offline. I'd encourage those who fall into this category to chime back in with their opinion rather than me asserting an opinion for them.

        Show
        Josh Elser added a comment - Sorry to hear that – how much contribution would you say you've been doing? The few patches I've made have been coordinated with Eric Newton. Some on jira, some off. Do you have any context on this? Like an email thread on list where this was discussed? No, most of this was discussed offline. I'd encourage those who fall into this category to chime back in with their opinion rather than me asserting an opinion for them.
        Hide
        Billie Rinaldi added a comment -

        I think it makes sense to move wikisearch to a contrib directory where it can evolve and be packaged independently of the core codebase. We can decide later if we want to include it in core releases, or if we want to release it separately. If people want to work on it, we can add them as committers. There seems to be a perception that it is more difficult to become a committer than it actually is. Perhaps we need to do better with our welcome wagon.

        Show
        Billie Rinaldi added a comment - I think it makes sense to move wikisearch to a contrib directory where it can evolve and be packaged independently of the core codebase. We can decide later if we want to include it in core releases, or if we want to release it separately. If people want to work on it, we can add them as committers. There seems to be a perception that it is more difficult to become a committer than it actually is. Perhaps we need to do better with our welcome wagon.
        Hide
        Chris A. Mattmann added a comment -

        Great suggestion, Billie, +1.

        Show
        Chris A. Mattmann added a comment - Great suggestion, Billie, +1.
        Hide
        jv added a comment -

        +1 for contrib. Personally, I think it should have an independent release cycle, because it's codebase will be independent from releases.

        Show
        jv added a comment - +1 for contrib. Personally, I think it should have an independent release cycle, because it's codebase will be independent from releases.
        Hide
        Billie Rinaldi added a comment -

        Done: moved trunk/examples/wikisearch to contrib/wikisearch/trunk

        TODO: decide whether to move trunk/examples/simple to trunk/examples, or to leave it as a submodule of examples
        Originally I was in favor of moving it, but now I'm thinking it would just make merging unnecessarily difficult. Who knows, maybe we'll decide to create another examples submodule at some point.

        TODO: decide whether pom structure of wikisearch needs to be modified or not
        It builds as is, so this can be investigated at a later time.

        Show
        Billie Rinaldi added a comment - Done: moved trunk/examples/wikisearch to contrib/wikisearch/trunk TODO: decide whether to move trunk/examples/simple to trunk/examples, or to leave it as a submodule of examples Originally I was in favor of moving it, but now I'm thinking it would just make merging unnecessarily difficult. Who knows, maybe we'll decide to create another examples submodule at some point. TODO: decide whether pom structure of wikisearch needs to be modified or not It builds as is, so this can be investigated at a later time.
        Hide
        Josh Elser added a comment -

        Billie, since you did the work to pull out of trunk and into contrib, can we close this?

        We can make new ticket(s) to address the TODOs you left.

        Show
        Josh Elser added a comment - Billie, since you did the work to pull out of trunk and into contrib, can we close this? We can make new ticket(s) to address the TODOs you left.
        Hide
        Billie Rinaldi added a comment -

        Sounds good, let's open other tickets for additional work.

        Show
        Billie Rinaldi added a comment - Sounds good, let's open other tickets for additional work.
        Billie Rinaldi made changes -
        Status Open [ 1 ] Resolved [ 5 ]
        Resolution Fixed [ 1 ]

          People

          • Assignee:
            Billie Rinaldi
            Reporter:
            Billie Rinaldi
          • Votes:
            1 Vote for this issue
            Watchers:
            11 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development