Details

    • Type: Umbrella Umbrella
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.99.2
    • Component/s: None
    • Labels:
      None

      Description

      Tasks for 1.0 copied here from our '1.0.0' mailing list discussion. Idea is to file subtasks off this one.

        Issue Links

          Activity

          Hide
          Enis Soztutar added a comment -

          Closing this issue after 0.99.2 release.

          Show
          Enis Soztutar added a comment - Closing this issue after 0.99.2 release.
          Hide
          stack added a comment -

          Resolving. Only outstanding subtasks are the update of jars in time for release 1.0 in mvn and our dependencies. That is on going. No other dependent issues (those not being worked on were moved out). I believe I covered all the outstanding doc issues in here and pushed the doc out.

          Show
          stack added a comment - Resolving. Only outstanding subtasks are the update of jars in time for release 1.0 in mvn and our dependencies. That is on going. No other dependent issues (those not being worked on were moved out). I believe I covered all the outstanding doc issues in here and pushed the doc out.
          Hide
          stack added a comment -

          Removed " HBASE-10403 Simplify offheap cache configuration" link as it has fix version 2.0.
          Ditto for "HBASE-10504 Define Replication Interface"

          Show
          stack added a comment - Removed " HBASE-10403 Simplify offheap cache configuration" link as it has fix version 2.0. Ditto for " HBASE-10504 Define Replication Interface"
          Hide
          stack added a comment -

          I also remove HBASE-5827 "[Coprocessors] Observer notifications on exceptions" as a link to this issue.

          Show
          stack added a comment - I also remove HBASE-5827 " [Coprocessors] Observer notifications on exceptions" as a link to this issue.
          Hide
          stack added a comment -

          I removed links from HBASE-11122 "Annotate coprocessor APIs", HBASE-11124 "Pluggable major compaction strategy", HBASE-9864 "Notifications bus for use by cluster members keeping up-to-date on changes" because not being worked on (to best of my knowledge – at least not in time for a 1.0 – correct me if wrong).

          I also removed HBASE-11125 "Introduce a higher level interface for registering interest in coprocessor upcalls" though it still has fix version of 0.99.2 so it is still related to 1.0.

          Show
          stack added a comment - I removed links from HBASE-11122 "Annotate coprocessor APIs", HBASE-11124 "Pluggable major compaction strategy", HBASE-9864 "Notifications bus for use by cluster members keeping up-to-date on changes" because not being worked on (to best of my knowledge – at least not in time for a 1.0 – correct me if wrong). I also removed HBASE-11125 "Introduce a higher level interface for registering interest in coprocessor upcalls" though it still has fix version of 0.99.2 so it is still related to 1.0.
          Hide
          stack added a comment -

          We can add a section for zookeeper similar to hadoop versions and jdk versions I guess.

          Yes. And call it out in the 0.98 to 1.0 upgrade section.

          DOCUMENTATION: Add doc around new feature HBASE-12286 et al. forward port by Manukranth Kolloju of 0.89fb online config change feature; list the configs that can be changed.

          Show
          stack added a comment - We can add a section for zookeeper similar to hadoop versions and jdk versions I guess. Yes. And call it out in the 0.98 to 1.0 upgrade section. DOCUMENTATION: Add doc around new feature HBASE-12286 et al. forward port by Manukranth Kolloju of 0.89fb online config change feature; list the configs that can be changed.
          Hide
          Enis Soztutar added a comment -

          DOCUMENTATION: New minimum zk required and useMulti on by default. See HBASE-12241

          We can add a section for zookeeper similar to hadoop versions and jdk versions I guess.

          Show
          Enis Soztutar added a comment - DOCUMENTATION: New minimum zk required and useMulti on by default. See HBASE-12241 We can add a section for zookeeper similar to hadoop versions and jdk versions I guess.
          Hide
          stack added a comment -

          DOCUMENTATION: New minimum zk required and useMulti on by default. See HBASE-12241

          Show
          stack added a comment - DOCUMENTATION: New minimum zk required and useMulti on by default. See HBASE-12241
          Hide
          stack added a comment -

          DOCUMENTATION: HBASE-12103 is about adding back v1 hfile checker.

          Show
          stack added a comment - DOCUMENTATION: HBASE-12103 is about adding back v1 hfile checker.
          Hide
          stack added a comment -

          DOCUMENTATION: Talk up filter perf issue that comes of transform(KV) and transformCell. See HBASE-12068

          Show
          stack added a comment - DOCUMENTATION: Talk up filter perf issue that comes of transform(KV) and transformCell. See HBASE-12068
          Hide
          Enis Soztutar added a comment -

          TODO: Test a migration going from 0.94 to 1.0. Would be great if it worked (or if only a little work needed to make it work)

          Created sub task: HBASE-11915

          Show
          Enis Soztutar added a comment - TODO: Test a migration going from 0.94 to 1.0. Would be great if it worked (or if only a little work needed to make it work) Created sub task: HBASE-11915
          Hide
          stack added a comment -

          TODO: Test a migration going from 0.94 to 1.0. Would be great if it worked (or if only a little work needed to make it work). So for now, lets not do anything that would for sure break this possiblity: e.g. HBASE-11333

          Show
          stack added a comment - TODO: Test a migration going from 0.94 to 1.0. Would be great if it worked (or if only a little work needed to make it work). So for now, lets not do anything that would for sure break this possiblity: e.g. HBASE-11333
          Hide
          stack added a comment -

          Consider this for 1.0 (caveat the question by Ishan on the end of the issue)

          Show
          stack added a comment - Consider this for 1.0 (caveat the question by Ishan on the end of the issue)
          Hide
          Enis Soztutar added a comment -

          I think 1.0 should be rolling upgradable from 0.98 unless we break it intentionally for a specific reason. Unless there is such an issue, lets document that 1.0 and 0.98 should be rolling upgrade compatible. The only possible offender from what I see so far is HBASE-10569. I'll create a separate subtask for documenting and testing it.

          Show
          Enis Soztutar added a comment - I think 1.0 should be rolling upgradable from 0.98 unless we break it intentionally for a specific reason. Unless there is such an issue, lets document that 1.0 and 0.98 should be rolling upgrade compatible. The only possible offender from what I see so far is HBASE-10569 . I'll create a separate subtask for documenting and testing it.
          Hide
          Enis Soztutar added a comment -

          You may have noticed that I started linking issues that we want to have before 1.0 to this issue. It does not mean that we will require every subtask or linked issue to be complete, but it is a good idea to aggregate important issues that we see we should resolve before 1.0. Please feel free to link, or pick up sub-tasks or linked issues.

          Show
          Enis Soztutar added a comment - You may have noticed that I started linking issues that we want to have before 1.0 to this issue. It does not mean that we will require every subtask or linked issue to be complete, but it is a good idea to aggregate important issues that we see we should resolve before 1.0. Please feel free to link, or pick up sub-tasks or linked issues.
          Hide
          stack added a comment -

          Enis Soztutar and Nick Dimiduk Thanks lads.

          Show
          stack added a comment - Enis Soztutar and Nick Dimiduk Thanks lads.
          Hide
          Enis Soztutar added a comment -

          I think Hive uses the mapred APIs even with h2.

          Show
          Enis Soztutar added a comment - I think Hive uses the mapred APIs even with h2.
          Hide
          Nick Dimiduk added a comment -
          Show
          Nick Dimiduk added a comment - Hadoop-2.3 shipped with the mapred API... http://hadoop.apache.org/docs/r2.3.0/api/org/apache/hadoop/mapred/package-summary.html
          Hide
          stack added a comment -

          + Chatting today, shouldn't we remove the mapred package seeing as 1.0 only supports 2.0 hadoop?

          Show
          stack added a comment - + Chatting today, shouldn't we remove the mapred package seeing as 1.0 only supports 2.0 hadoop?
          Hide
          Andrew Purtell added a comment -

          Definitely agree about HBASE-4047. The main reason I haven't tried to tackle that yet is I estimate ~3 months to get it off the ground.

          Show
          Andrew Purtell added a comment - Definitely agree about HBASE-4047 . The main reason I haven't tried to tackle that yet is I estimate ~3 months to get it off the ground.
          Hide
          stack added a comment -

          Adding this one until it gets punted.

          Show
          stack added a comment - Adding this one until it gets punted.
          Hide
          stack added a comment -

          I really doubt some of the items will make in 1.0

          Agree. I did a comprehensive list first of what folks wanted up in the ML. Feel free to edit. Would be good to call out the must-haves.

          Show
          stack added a comment - I really doubt some of the items will make in 1.0 Agree. I did a comprehensive list first of what folks wanted up in the ML. Feel free to edit. Would be good to call out the must-haves.
          Hide
          Enis Soztutar added a comment -

          Thanks Stack for creating this. Let's start linking the issues.
          From the above list, I really doubt some of the items will make in 1.0 (like HBASE-4047), but some of them are absolute must if you ask me. Especially, we should be focussing on the APIs a bit, and some correctness fixes (seqId-mvcc, dist log replay depends on this, and also HBASE-9905).

          Show
          Enis Soztutar added a comment - Thanks Stack for creating this. Let's start linking the issues. From the above list, I really doubt some of the items will make in 1.0 (like HBASE-4047 ), but some of them are absolute must if you ask me. Especially, we should be focussing on the APIs a bit, and some correctness fixes (seqId-mvcc, dist log replay depends on this, and also HBASE-9905 ).
          Hide
          stack added a comment -

          Above Hudson message should have showed in HBASE-10888 (I had wrong svn log message, wrong issue referenced, since corrected)

          Show
          stack added a comment - Above Hudson message should have showed in HBASE-10888 (I had wrong svn log message, wrong issue referenced, since corrected)
          Hide
          Hudson added a comment -

          FAILURE: Integrated in HBase-TRUNK #5061 (See https://builds.apache.org/job/HBase-TRUNK/5061/)
          HBASE-10856 Enable distributed log replay as default (stack: rev 1584346)

          • /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java
          • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
          • /hbase/trunk/hbase-common/src/main/resources/hbase-default.xml
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/DeadServer.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/LogReplayHandler.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
          Show
          Hudson added a comment - FAILURE: Integrated in HBase-TRUNK #5061 (See https://builds.apache.org/job/HBase-TRUNK/5061/ ) HBASE-10856 Enable distributed log replay as default (stack: rev 1584346) /hbase/trunk/hbase-client/src/main/java/org/apache/hadoop/hbase/zookeeper/RecoverableZooKeeper.java /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java /hbase/trunk/hbase-common/src/main/resources/hbase-default.xml /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/DeadServer.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/SplitLogManager.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/CreateTableHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/handler/LogReplayHandler.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/wal/HLogSplitter.java
          Hide
          stack added a comment -

          DOCUMENTATION: Of note, you cannot (safely) rolling restart from 0.96 on to 0.99. See HBASE-9801 where if the rolling upgrade fails, you could have old code trying to open hfile v3s written w/ new code.

          Show
          stack added a comment - DOCUMENTATION: Of note, you cannot (safely) rolling restart from 0.96 on to 0.99. See HBASE-9801 where if the rolling upgrade fails, you could have old code trying to open hfile v3s written w/ new code.
          Hide
          stack added a comment - - edited

          Here is the thread: http://search-hadoop.com/m/33P7E1GL08b

          Here are items. I'll file issues for them:

          • Update included libs (e.g. move to log4j2)
          • Enable distributed log replay as default (fix bugs)
          • Enable hfilev3 as default.
          • Ship with default logging level set to INFO and content of the logs still makes sense
          • Enable dynamic config and schema by default.
          • Maybe make the hconnection output a bit less verbose when ZK isn't live or HBase still coming up; it does generate a fair amount of log data
          • Light pastel purple colour scheme.
          • @Deprecated APIs cleaned up. Delete stale methods or remove the annotations.
          • Fix interface audience annotations (HBASE-10462).
          • Continue on limiting some broad reaching interfaces (HCM, HCI, HTable) See recent HBASE-10479 discussions)
          • Promote HTableInterface vs HTable, getting connections from HCM and getting tables there.
          • A clearly defined compatibility strategy between patch, minor, and major versions
          • Known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface.
          • (wishes for ponies and unicorns, probably not going to happen) better ops supports, such as sample puppet/chef/etc scripts
          • HBASE-4047. (Maybe that makes the 1.0 list?)
          • Paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins... lots of discussion here including ..."I think the best place to start is by breaking up some of the current APIs, grouping them around behaviors or areas of functionality. "
          • In HRegion ....extract the public interface out into a Interface. (similar to Store and HStore)
          • HBASE-8763 Combine MVCC and SeqId. Currently replication is broken on same version updates in the following scenarios: when a region move or RS get restarted in the source cluster, replicated changes may come out of order to the receiving RS. There are other cases we need to keep the ordering of puts. This JIRA can be used to fix those above issues.
          • Replicated Meta Regions for Distributed Region Location Lookup(I haven't cut a JIRA for it but I've been thinking this for a while)
          • "We need a Replication Interface defined"
          • HBASE-9206 'namespace permissions'
          • c-client
          Show
          stack added a comment - - edited Here is the thread: http://search-hadoop.com/m/33P7E1GL08b Here are items. I'll file issues for them: Update included libs (e.g. move to log4j2) Enable distributed log replay as default (fix bugs) Enable hfilev3 as default. Ship with default logging level set to INFO and content of the logs still makes sense Enable dynamic config and schema by default. Maybe make the hconnection output a bit less verbose when ZK isn't live or HBase still coming up; it does generate a fair amount of log data Light pastel purple colour scheme. @Deprecated APIs cleaned up. Delete stale methods or remove the annotations. Fix interface audience annotations ( HBASE-10462 ). Continue on limiting some broad reaching interfaces (HCM, HCI, HTable) See recent HBASE-10479 discussions) Promote HTableInterface vs HTable, getting connections from HCM and getting tables there. A clearly defined compatibility strategy between patch, minor, and major versions Known coprocessor interfaces. For example right now a coprocessor can use everything on HRegion. We need to distill what's useful into an interface. (wishes for ponies and unicorns, probably not going to happen) better ops supports, such as sample puppet/chef/etc scripts HBASE-4047 . (Maybe that makes the 1.0 list?) Paring down the coprocessor interfaces to only intercept RPC based actions and move everything else to plugins... lots of discussion here including ..."I think the best place to start is by breaking up some of the current APIs, grouping them around behaviors or areas of functionality. " In HRegion ....extract the public interface out into a Interface. (similar to Store and HStore) HBASE-8763 Combine MVCC and SeqId. Currently replication is broken on same version updates in the following scenarios: when a region move or RS get restarted in the source cluster, replicated changes may come out of order to the receiving RS. There are other cases we need to keep the ordering of puts. This JIRA can be used to fix those above issues. Replicated Meta Regions for Distributed Region Location Lookup(I haven't cut a JIRA for it but I've been thinking this for a while) "We need a Replication Interface defined" HBASE-9206 'namespace permissions' c-client

            People

            • Assignee:
              Unassigned
              Reporter:
              stack
            • Votes:
              0 Vote for this issue
              Watchers:
              20 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development