Mahout
  1. Mahout
  2. MAHOUT-999

KMeans failing to create correct Clustering Policy

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.7
    • Component/s: None
    • Labels:
      None

      Description

      Jenkins has been complaining about the cluster-reuters.sh example for a while and the issue appears to be due to trying to write a file to a directory that doesn't exist due to the fact that it is a file, not a directory.

      
      Exception in thread "main" java.io.IOException: Mkdirs failed to create /tmp/mahout-work-grantingersoll/reuters-kmeans-clusters/part-randomSeed/clusters-0
      	at org.apache.hadoop.fs.ChecksumFileSystem.create(ChecksumFileSystem.java:366)
      	at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:528)
      	at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:843)
      	at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:831)
      	at org.apache.hadoop.io.SequenceFile$Writer.<init>(SequenceFile.java:823)
      	at org.apache.mahout.clustering.classify.ClusterClassifier.writePolicy(ClusterClassifier.java:232)
      	at org.apache.mahout.clustering.classify.ClusterClassifier.writeToSeqFiles(ClusterClassifier.java:185)
      	at org.apache.mahout.clustering.kmeans.KMeansDriver.buildClusters(KMeansDriver.java:254)
      	at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:154)
      	at org.apache.mahout.clustering.kmeans.KMeansDriver.run(KMeansDriver.java:104)
      	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
      	at org.apache.mahout.clustering.kmeans.KMeansDriver.main(KMeansDriver.java:48)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      	at java.lang.reflect.Method.invoke(Method.java:597)
      	at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:68)
      	at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:139)
      	at org.apache.mahout.driver.MahoutDriver.main(MahoutDriver.java:188)
      

      Run examples/bin/cluster-reuters.sh and choose 1 (KMeans) to see the error.

        Activity

        Grant Ingersoll created issue -
        Hide
        Paritosh Ranjan added a comment -

        Will try to fix it soon. Running really short of time. Will do this first thing as soon as I get time.
        Thanks for the debugging and Jira Issue.

        Show
        Paritosh Ranjan added a comment - Will try to fix it soon. Running really short of time. Will do this first thing as soon as I get time. Thanks for the debugging and Jira Issue.
        Paritosh Ranjan made changes -
        Field Original Value New Value
        Assignee Paritosh Ranjan [ paritoshranjan ]
        Paritosh Ranjan made changes -
        Status Open [ 1 ] In Progress [ 3 ]
        Hide
        Paritosh Ranjan added a comment -

        Fixed it for kmeans as will as fuzzykmeans. The current example clusters build is demonstrating that.

        Now creating directories to write for Clustering Policies.

        Show
        Paritosh Ranjan added a comment - Fixed it for kmeans as will as fuzzykmeans. The current example clusters build is demonstrating that. Now creating directories to write for Clustering Policies.
        Paritosh Ranjan made changes -
        Status In Progress [ 3 ] Resolved [ 5 ]
        Fix Version/s 0.7 [ 12319261 ]
        Resolution Fixed [ 1 ]
        Sean Owen made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Paritosh Ranjan
            Reporter:
            Grant Ingersoll
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development