Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10150

Hadoop cryptographic file system



    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: 2.6.0
    • Component/s: security
    • Labels:


      There is an increasing need for securing data when Hadoop customers use various upper layer applications, such as Map-Reduce, Hive, Pig, HBase and so on.

      HADOOP CFS (HADOOP Cryptographic File System) is used to secure data, based on HADOOP “FilterFileSystem” decorating DFS or other file systems, and transparent to upper layer applications. It’s configurable, scalable and fast.

      High level requirements:
      1. Transparent to and no modification required for upper layer applications.
      2. “Seek”, “PositionedReadable” are supported for input stream of CFS if the wrapped file system supports them.
      3. Very high performance for encryption and decryption, they will not become bottleneck.
      4. Can decorate HDFS and all other file systems in Hadoop, and will not modify existing structure of file system, such as namenode and datanode structure if the wrapped file system is HDFS.
      5. Admin can configure encryption policies, such as which directory will be encrypted.
      6. A robust key management framework.
      7. Support Pread and append operations if the wrapped file system supports them.


        1. cfs.patch
          104 kB
          Yi Liu
        2. CryptographicFileSystem.patch
          287 kB
          Yi Liu
        3. extended information based on INode feature.patch
          128 kB
          Yi Liu
        4. HADOOP cryptographic file system.pdf
          561 kB
          Yi Liu
        5. HADOOP cryptographic file system-V2.docx
          103 kB
          Yi Liu
        6. HDFSDataAtRestEncryptionAlternatives.pdf
          321 kB
          Alejandro Abdelnur
        7. HDFSDataatRestEncryptionAttackVectors.pdf
          131 kB
          Alejandro Abdelnur
        8. HDFSDataatRestEncryptionProposal.pdf
          219 kB
          Alejandro Abdelnur

          Issue Links

          There are no Sub-Tasks for this issue.



              • Assignee:
                hitliuyi Yi Liu
                hitliuyi Yi Liu
              • Votes:
                0 Vote for this issue
                62 Start watching this issue


                • Created: