Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14478

Add libhdfs APIs for openFile

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: hdfs-client, libhdfs, native
    • Labels:
      None

      Description

      HADOOP-15229 added a "FileSystem builder-based openFile() API" that allows specifying configuration values for opening files (similar to HADOOP-14365).

      Support for openFile will be a little tricky as it is asynchronous and FutureDataInputStreamBuilder#build returns a CompletableFuture.

      At a high level, the API for openFile could look something like this:

      hdfsFile hdfsOpenFile(hdfsFS fs, const char* path, int flags,
                            int bufferSize, short replication, tSize blocksize);
      
      hdfsOpenFileBuilder *hdfsOpenFileBuilderAlloc(hdfsFS fs,
              const char *path);
      
      hdfsOpenFileBuilder *hdfsOpenFileBuilderMust(hdfsOpenFileBuilder *builder,
              const char *key, const char *value);
      
      hdfsOpenFileBuilder *hdfsOpenFileBuilderOpt(hdfsOpenFileBuilder *builder,
              const char *key, const char *value);
      
      hdfsOpenFileFuture *hdfsOpenFileBuilderBuild(hdfsOpenFileBuilder *builder);
      
      void hdfsOpenFileBuilderFree(hdfsOpenFileBuilder *builder);
      
      hdfsFile hdfsOpenFileFutureGet(hdfsOpenFileFuture *future);
      
      hdfsFile hdfsOpenFileFutureGetWithTimeout(hdfsOpenFileFuture *future,
              int64_t timeout, javaConcurrentTimeUnit timeUnit);
      
      int hdfsOpenFileFutureCancel(hdfsOpenFileFuture *future,
              int mayInterruptIfRunning);
      
      void hdfsOpenFileFutureFree(hdfsOpenFileFuture *future);
      
      

      Instead of exposing all the functionality of CompleteableFuture libhdfs would just expose the functionality of Future.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                stakiar Sahil Takiar
                Reporter:
                stakiar Sahil Takiar
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated: