Cassandra
  1. Cassandra
  2. CASSANDRA-5185

symlinks to data directories are broken in 1.2.0

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Fix Version/s: 1.2.2
    • Component/s: Core
    • Labels:
      None

      Description

      symlinks to data directories is broken in 1.2.0

      cd ~
      tar xzf apache-cassandra-1.2.0-bin.tar.gz
      cd apache-cassandra-1.2.0/conf
      vim cassandra.yaml
      #set data/commitlog/savecache dirs to
      #~/apache-cassandra-1.2.0/var/...
      cd ../bin
      # start once to make folders
      ./cassandra -f
      #cntrl-c
      cd ..
      mkdir var/lib/cassandra2
      mv var/lib/cassandra/data/system var/lib/cassandra2/system
      cd var/lib/cassandra/data
      ln -s ../../cassandra2/system .
      cd ~/apache-cassandra-1.2.0/bin/
      cassandra -f
      #get lots of assertion errors see attached log file
      
       INFO 21:59:44,883 Enqueuing flush of Memtable-local@1578022692(52/52 serialized/live bytes, 2 ops)
       INFO 21:59:44,890 Writing Memtable-local@1578022692(52/52 serialized/live bytes, 2 ops)
      ERROR 21:59:44,892 Exception in thread Thread[FlushWriter:1,5,main]
      java.lang.AssertionError
      	at org.apache.cassandra.io.sstable.Descriptor.<init>(Descriptor.java:190)
      	at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:593)
      	at org.apache.cassandra.db.ColumnFamilyStore.getTempSSTablePath(ColumnFamilyStore.java:588)
      	at org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:428)
      	at org.apache.cassandra.db.Memtable$FlushRunnable.runWith(Memtable.java:417)
      	at org.apache.cassandra.io.util.DiskAwareRunnable.runMayThrow(DiskAwareRunnable.java:48)
      	at org.apache.cassandra.utils.WrappedRunnable.run(WrappedRunnable.java:28)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
      	at java.lang.Thread.run(Thread.java:662)
      

      I traced it back some, I think it is coming from:

      org.apache.cassandra.db.Memtable$FlushRunnable.writeSortedContents(Memtable.java:428)
      which calls: cfs.directories.getLocationForDisk(dataDirectory)
      
          public File getLocationForDisk(File dataDirectory)
          {
              for (File dir : sstableDirectories)
              {
                  if (FileUtils.getCanonicalPath(dir).startsWith(FileUtils.getCanonicalPath(dataDirectory)))
                      return dir;
              }
              return null;
          }
      

      My guess is that the FileUtils.getCanonicalPath calls aren't matching because of the symlinks. So null is being returned there.

      1. 5185-errors.log
        8 kB
        Jeremiah Jordan
      2. 5185.txt
        0.6 kB
        Yuki Morishita

        Activity

        Hide
        Yuki Morishita added a comment -

        File#getCannonicalPath returns path after resolving symlinks. Patch instead uses getAbsolutePath so symlinks under data dir works as expected.

        Show
        Yuki Morishita added a comment - File#getCannonicalPath returns path after resolving symlinks. Patch instead uses getAbsolutePath so symlinks under data dir works as expected.
        Hide
        Jonathan Ellis added a comment -

        If we cannonicalize both sizes of what we're comparing, shouldn't it work either way?

        Show
        Jonathan Ellis added a comment - If we cannonicalize both sizes of what we're comparing, shouldn't it work either way?
        Hide
        Yuki Morishita added a comment -

        For example, if you have data dir of "/var/lib/cassandra/data" and you symlinked system keyspace under that data dir to "/var/lib/cassandra2/system" as above, cannonicalized paths are

        /var/lib/cassandra/data -> /var/lib/cassandra/data
        /var/lib/cassandra/data/system -> /var/lib/cassandra2/system
        

        thus current comparison doesn't work.

        Show
        Yuki Morishita added a comment - For example, if you have data dir of "/var/lib/cassandra/data" and you symlinked system keyspace under that data dir to "/var/lib/cassandra2/system" as above, cannonicalized paths are /var/lib/cassandra/data -> /var/lib/cassandra/data /var/lib/cassandra/data/system -> /var/lib/cassandra2/system thus current comparison doesn't work.
        Hide
        Jonathan Ellis added a comment -

        +1

        Show
        Jonathan Ellis added a comment - +1
        Hide
        Yuki Morishita added a comment -

        Committed.

        Show
        Yuki Morishita added a comment - Committed.

          People

          • Assignee:
            Yuki Morishita
            Reporter:
            Jeremiah Jordan
            Reviewer:
            Jonathan Ellis
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development