Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-6677

Not able to access the Ozone cluster running in Kuberenets through hdfs dfs -put/cat via hadoop client from external network

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • 1.2.1
    • None
    • kubernetes, Ozone Datanode
    • None

    Description

      Ozone cluster is hosted in Kubernetes cluster with 3 datanode and 1 OM/SCM/S3G pod.

      Cluster is created as per the steps mentioned in documentation and I am able to read/write files via hadoop client if its hosted with in Kubernetes cluster But I am not able to read/write the files using hadoop client which is hosted outside kuberenetes cluster. 

      There is no documentation available which defines about connectivity to ozone cluster from outside network.

      Getting below error. As per my initial investigation , hadoop client is trying to access directly the datanode pods but since no service has been defined with nodeport, it failed to communicate with it.

      2022-04-28 18:25:06,135 WARN io.KeyOutputStream: Encountered exception java.io.IOException: Unexpected Storage Container Exception: java.util.concurrent.CompletionException: Failed to write chunk 109611004723200005_chunk_1 into block conID: 5 locID: 109611004723200005 bcsId: 0 on the pipeline Pipeline[ Id: 2a523691-e3ee-4826-bf77-89a40a161194, Nodes: e837c443-7e06-4484-85be-8e36807030a1{ip: 10.42.4.0, host: 10.42.4.0, ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default-rack, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}73608fb4-7dcb-426b-8e71-0c7b9bc31271{ip: 10.42.1.0, host: 10.42.1.0, ports: [REPLICATION=9886, RATIS=9858, RATIS_ADMIN=9857, RATIS_SERVER=9856, STANDALONE=9859], networkLocation: /default-rack, certSerialId: null, persistedOpState: IN_SERVICE, persistedOpStateExpiryEpochSec: 0}, ReplicationConfig: RATIS/THREE, State:OPEN, leaderId:73608fb4-7dcb-426b-8e71-0c7b9bc31271, CreationTimestamp2022-04-28T18:05:45.383+05:30[Asia/Kolkata]]. The last committed block length is 0, uncommitted data length is 11 retry count 0

      ^C2022-04-28 18:25:25,616 WARN grpc.GrpcUtil: Timed out gracefully shutting down connection: ManagedChannelOrphanWrapper{delegate=ManagedChannelImpl{logId=41, target=10.42.4.0:9858}}.

       

       

       

      Let me know the procedure to access the cluster from external network.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            bgdileep Dileep

            Dates

              Created:
              Updated:

              Slack

                Issue deployment