Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
Reviewed
Description
I deployed hadoop-trunk HDFS and created /user/schu/. I then forced a checkpoint, fetched the fsimage, and ran the default OfflineImageViewer successfully on the fsimage.
schu-mbp:~ schu$ hdfs oiv -i fsimage_0000000000000000004 -o oiv_out_1 schu-mbp:~ schu$ cat oiv_out_1 drwxr-xr-x - schu supergroup 0 2013-05-24 16:59 / drwxr-xr-x - schu supergroup 0 2013-05-24 16:59 /user drwxr-xr-x - schu supergroup 0 2013-05-24 16:59 /user/schu schu-mbp:~ schu$
I then touched an empty file /user/schu/testFile1
schu-mbp:~ schu$ hadoop fs -lsr /
lsr: DEPRECATED: Please use 'ls -R' instead.
drwxr-xr-x - schu supergroup 0 2013-05-24 16:59 /user
drwxr-xr-x - schu supergroup 0 2013-05-24 17:00 /user/schu
-rw-r--r-- 1 schu supergroup 0 2013-05-24 17:00 /user/schu/testFile1
and forced another checkpoint, fetched the fsimage, and reran the OfflineImageViewer. I encountered a NegativeArraySizeException:
schu-mbp:~ schu$ hdfs oiv -i fsimage_0000000000000000008 -o oiv_out_2
Input ended unexpectedly.
2013-05-24 17:01:13,622 ERROR [main] offlineImageViewer.OfflineImageViewer (OfflineImageViewer.java:go(140)) - image loading failed at offset 402
Exception in thread "main" java.lang.NegativeArraySizeException
at org.apache.hadoop.io.Text.readString(Text.java:458)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processPermission(ImageLoaderCurrent.java:370)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processINode(ImageLoaderCurrent.java:671)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processChildren(ImageLoaderCurrent.java:557)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processDirectoryWithSnapshot(ImageLoaderCurrent.java:464)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processDirectoryWithSnapshot(ImageLoaderCurrent.java:470)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processDirectoryWithSnapshot(ImageLoaderCurrent.java:470)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processLocalNameINodesWithSnapshot(ImageLoaderCurrent.java:444)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.processINodes(ImageLoaderCurrent.java:398)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.ImageLoaderCurrent.loadImage(ImageLoaderCurrent.java:199)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewer.go(OfflineImageViewer.java:136)
at org.apache.hadoop.hdfs.tools.offlineImageViewer.OfflineImageViewer.main(OfflineImageViewer.java:260)
This is reproducible. I've reproduced this scenario after formatting HDFS and restarting and touching an empty file /testFile1.
Attached are the data dirs, the fsimage before creating the empty file (fsimage_0000000000000000004) and the fsimage afterwards (fsimage_0000000000000000004) and their outputs, oiv_out_1 and oiv_out_2 respectively.
The oiv_out_2 does not include the empty /user/schu/testFile1.
I don't run into this problem using hadoop-2.0.4-alpha.
