Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-6441

Ozone metadata does not align with underlying blocks when there are many incomplete uploads happens

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 1.2.0
    • None
    • Ozone Datanode
    • None

    Description

      Ozone metadata does not align with underlying blocks when there are many incomplete uploads happens. I have a cluster which has a very few objects. But the datanode usage tells me I almost run out of space.

      ????
      Usage info for datanode with UUID f50108f1-d8bf-44e3-abed-6e77c91f994d:
      Capacity : 8802545958912B
      SCMUsed : 8802128257024B (99.99525%)
      Remaining : 74715136B (0.00085%)

      Usage info for datanode with UUID 2bdb3198-b71f-4153-9663-e3b349c6f82a:
      Capacity : 8802545958912B
      SCMUsed : 8802133102592B (99.99531%)
      Remaining : 76824576B (0.00087%)

      Usage info for datanode with UUID d5644a36-b967-44a6-a736-4bd2013c2b86:
      Capacity : 8793955991552B
      SCMUsed : 8793311227904B (99.99267%)
      Remaining : 291676160B (0.00332%)
      ...
       

      Also I see there are lots of errors in logs, complaining out of disk space and also report missing .container files as below:

      ????
      2022-03-10 03:56:02 ERROR Thread-6 ContainerReader:159 - Missing .container file for ContainerID: 15221
       

      Attachments

        Issue Links

          Activity

            People

              erose Ethan Rose
              sokui Shawn
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: