Uploaded image for project: 'Kafka'
  1. Kafka
  2. KAFKA-16947

Kafka Tiered Storage V2

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • 4.0.0
    • None

    Attachments

      Issue Links

        is a parent of

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-16088 Not reading active segments when RemoteFetch return Empty Records.

        • Critical - Crashes, loss of data, severe memory leak.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-17184 Remote index cache noisy logging

        • Critical - Crashes, loss of data, severe memory leak.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-12641 Clear RemoteLogLeaderEpochState entry when it become empty.

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-13355 Shutdown broker eventually when unrecoverable exceptions like IOException encountered in RLMM.

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-15301 [Tiered Storage] Historically compacted topics send request to remote for active segment during consume

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-15341 Enabling TS for a topic during rolling restart causes problems

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-15678 [Tiered Storage] Stall remote reads with long-spanning transactions

        • Major - Major loss of function.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-15660 File-based Tiered Storage should delete folders upon topic deletion

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-15969 Align RemoteStorageThreadPool metrics name with KIP-405

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-15331 Handle remote log enabled topic deletion when leader is not available

        • Major - Major loss of function.
        • In Progress

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-16895 RemoteCopyLagSegments metric taking active segment into account

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Resolved

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-15388 Handle topics that were having compaction as retention earlier are changed to delete only retention policy and onboarded to tiered storage.

        • Major - Major loss of function.
        • Patch Available

        Bug - A problem which impairs or prevents the functions of the product. KAFKA-15931 Cached transaction index gets closed if tiered storage read is interrupted

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Patch Available

        New Feature - A new feature of the product, which has yet to be developed. KAFKA-15132 Implement disable & re-enablement for Tiered Storage

        • Major - Major loss of function.
        • Resolved

        New Feature - A new feature of the product, which has yet to be developed. KAFKA-16228 Add remote log metadata flag to the dump log tool

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-14915 Option to consume multiple partitions that have their data in remote storage for the target offsets.

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-15166 Add deletePartition API to the RemoteStorageManager

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-15195 Regenerate segment-aligned producer snapshots when upgrading to a Kafka version supporting Tiered Storage

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-15231 Add ability to pause/resume Remote Log Manager tasks

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-15313 Delete remote log segments partition asynchronously when a partition is deleted.

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-15864 Add more tests asserting the log-start-offset, local-log-start-offset, and HW/LSO/LEO in rolling over segments with tiered storage.

        • Major - Major loss of function.
        • Open

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-15094 Add RemoteIndexCache metrics like misses/evictions/load-failures.

        • Major - Major loss of function.
        • In Progress

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-15433 Follower fetch from tiered offset

        • Major - Major loss of function.
        • In Progress

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-15265 Remote copy/fetch quotas for tiered storage.

        • Major - Major loss of function.
        • Resolved

        Improvement - An improvement or enhancement to an existing feature or task. KAFKA-15434 Tiered Storage Quotas

        • Major - Major loss of function.
        • Resolved

        Test - A new unit, integration or system test. KAFKA-15529 Flaky test ReassignReplicaShrinkTest.executeTieredStorageTest

        • Blocker - Blocks development and/or testing work, production could not run
        • Open

        Test - A new unit, integration or system test. KAFKA-9578 Kafka Tiered Storage - System Tests

        • Major - Major loss of function.
        • Open

        Test - A new unit, integration or system test. KAFKA-15772 Flaky test TransactionsWithTieredStoreTest

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        Task - A task that needs to be done. KAFKA-13560 Load indexes and data in async manner in the critical path of replica fetcher threads.

        • Major - Major loss of function.
        • Open

        Task - A task that needs to be done. KAFKA-15300 Include remotelog size in complete log size and also add local log size and remote log size separately in kafka-log-dirs tool.

        • Major - Major loss of function.
        • Open

        Task - A task that needs to be done. KAFKA-15431 Add support to assert offloaded segment for already produced event in Tiered Storage Framework

        • Major - Major loss of function.
        • Open

        Task - A task that needs to be done. KAFKA-15480 Add RemoteStorageInterruptedException

        • Major - Major loss of function.
        • Open

        Task - A task that needs to be done. KAFKA-15682 Ensure internal remote log metadata topic does not expire its segments before deleting user-topic segments

        • Major - Major loss of function.
        • Open

        Task - A task that needs to be done. KAFKA-16454 Snapshot the state of remote log metadata for all the partitions

        • Major - Major loss of function.
        • Open

        Task - A task that needs to be done. KAFKA-15214 Add metrics for OffsetOutOfRangeException when tiered storage is enabled

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Open

        Task - A task that needs to be done. KAFKA-16780 Txn consumer exerts pressure on remote storage when collecting aborted transactions

        • Major - Major loss of function.
        • In Progress

        Task - A task that needs to be done. KAFKA-15376 Explore options of removing data earlier to the current leader's leader epoch lineage for topics enabled with tiered storage.

        • Major - Major loss of function.
        • Reopened

        Task - A task that needs to be done. KAFKA-15859 Introduce delayed remote list offsets to make LIST_OFFSETS async

        • Major - Major loss of function.
        • Resolved

        Task - A task that needs to be done. KAFKA-16976 Improve the dynamic config handling for RemoteLogManagerConfig when a broker is restarted.

        • Major - Major loss of function.
        • Resolved

        Task - A task that needs to be done. KAFKA-14467 Add a test to validate the replica state after processing the OFFSET_MOVED_TO_TIERED_STORAGE error, especially for the transactional state

        • Major - Major loss of function.
        • Patch Available

        Task - A task that needs to be done. KAFKA-15038 Use topic id/name mapping from the Metadata cache in the RemoteLogManager

        • Minor - Minor loss of function, or other problem where easy workaround is present.
        • Patch Available
        mentioned in

        Page Loading...

        Activity

          People

            satish.duggana Satish Duggana
            satish.duggana Satish Duggana
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated: