Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6581 Write to single replica in memory
  3. HDFS-6929

NN periodically unlinks lazy persist files with missing replicas from namespace

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.6.0
    • Component/s: datanode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Occasional data loss is expected when using the lazy persist flag due to node restarts. The NN will optionally unlink lazy persist files from the namespace to avoid them from showing up as corrupt files.

      This behavior can be turned off with a global option. In the future this may be made a per-file option controllable by the client.

        Attachments

        1. HDFS-6929.01.patch
          9 kB
          Arpit Agarwal
        2. HDFS-6929.02.patch
          10 kB
          Arpit Agarwal

          Issue Links

            Activity

              People

              • Assignee:
                arp Arpit Agarwal
                Reporter:
                arp Arpit Agarwal
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: