Uploaded image for project: 'Kudu'
  1. Kudu
  2. KUDU-3344

Master could do some garbage collection work in CatalogManagerBgTasks thread

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Minor
    • Resolution: Fixed
    • None
    • 1.17.0
    • master
    • None

    Description

      Kudu master now reserve all tables/tablets' metadata in memory and disks, deleted tables and tablets were marked REMOVED/DELETED/REPLACED state but not really deleted. This could lead to huge memory usage as described in KUDU-3097.  

      I think it's a good idea to cleanup them in the CatalogManagerBgTasks thread. But because the data deletion tasks are done asynchronously by tablet servers, it is uncertain when metadata can be safely deleted.

      Besides, we could also cleanup dead tablet servers from master's in-memory map in this thread, as I mentioned in KUDU-2915.

      Attachments

        Activity

          People

            Unassigned Unassigned
            zhangyifan27 YifanZhang
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: