comment lines in the cookies.txt file cause an exception and the rest of the file is ignored (should generally report and skip invalid lines and continue):
2019-07-25 16:58:24,052 WARN http.Http - Failed to read http.agent.host.cookie.file cookies.txt: java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.nutch.protocol.http.api.HttpBase.setConf(HttpBase.java:278)
could add "http.agent.host.cookie.file" to nutch-default.xml
Sebastian Nagel
added a comment - Hi markus17 looks good and works. A few minor points:
converting the URL object to a String, then parsing it again doesn't seem efficient (could just pass the URL object itself):
cookie = http.getCookie(url.toString());
...
public String getCookie( String url) {
if (hostCookies != null ) {
return hostCookies.get(URLUtil.getHost(url));
}
...
comment lines in the cookies.txt file cause an exception and the rest of the file is ignored (should generally report and skip invalid lines and continue):
2019-07-25 16:58:24,052 WARN http.Http - Failed to read http.agent.host.cookie.file cookies.txt: java.lang.ArrayIndexOutOfBoundsException: 1
at org.apache.nutch.protocol.http.api.HttpBase.setConf(HttpBase.java:278)
could add "http.agent.host.cookie.file" to nutch-default.xml
Hi markus17 looks good and works. A few minor points: