[NUTCH-1461] Problem with TableUtil

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Auto Closed
• Affects Version/s: nutchgora
• Fix Version/s: 2.5
• Component/s: parser

Environment

Debian / CDH3 / Nutch 2.0 Release

• Patch Info: Patch Available
• Assignee: Unassigned
• Reporter: Christian Johnsson
• Created: 28/Aug/12 00:53
• Updated: 13/Oct/19 22:36
• Resolved: 13/Oct/19 22:36

Description

Affects parse and updatedb and parse.

Think i got some missformated urls into hbase but i can't fin them.
It generates this error though. If i empty hbase and restart it goes for a couple of million pages indexed then it comes up again. Any tips on how to locate what row in the table that genereates this error?

2012-08-28 01:48:10,871 WARN org.apache.hadoop.mapred.Child: Error running child
java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.nutch.util.TableUtil.unreverseUrl(TableUtil.java:98)
at org.apache.nutch.parse.ParserJob$ParserMapper.map(ParserJob.java:102)
at org.apache.nutch.parse.ParserJob$ParserMapper.map(ParserJob.java:76)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:647)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:323)
at org.apache.hadoop.mapred.Child$4.run(Child.java:266)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1278)
at org.apache.hadoop.mapred.Child.main(Child.java:260)
2012-08-28 01:48:10,875 INFO org.apache.hadoop.mapred.Task: Runnning cleanup for the task

Issue Links

• is duplicated by: NUTCH-1448 Redirected urls should be handled more cleanly (more like an outlink url)