Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10252

Is DataNode aware of the name of the file that it is going to store?

    XMLWordPrintableJSON

Details

    • Test
    • Status: Resolved
    • Minor
    • Resolution: Invalid
    • None
    • None
    • datanode, namenode
    • None

    Description

      I am going through the HDFS Namenode and Datanode code and I am trying to see if the DataNode is aware of the names of the files that are stored in it (and other metadata as well).

      Assuming that we have the most simple case:
      1 NameNode
      1 DataNode
      1 single machine running HDFS with replication factor 1.

      and considering the way HDFS works a use case could be:
      A client requests to write a file from local to HDFS (for example: "hdfs dfs -put file /file")
      He first communicates with NameNode and gets where this file should be stored.
      Then, after receiving an answer, he requests to the DataNode to store that file.

      (At that point I am going to be a little more specific about the code)
      The DataNode has a DataXceiverServer class which runs and waits for requests. When a request comes, it starts a DataXceiver thread and try to serve that request. What I would like to know is, if at that specific point the DataNode knows the name of the file that it is going to store. I spent hours of debugging but I could not find it. Is it somewhere there, or only the NameNode knows the name of that file?

      Attachments

        Activity

          People

            Unassigned Unassigned
            sarigian Dino Saur
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: