[CONNECTORS-1569] IBM WebSEAL authentication

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: ManifoldCF 2.11
• Component/s: Web connector
• Labels: ManifoldCF

Environment

Manifold 2.11
IBM WebSEAL

• Assignee: Karl Wright
• Reporter: Ferdi Klomp
• Created: 10/Jan/19 11:28
• Updated: 10/Jan/19 15:57

Description

Hi,

We have stumbled upon a problem with the Web Connector authentication in relation to IBM WebSEAL. We were unable to perform a successfully authentication against WebSEAL. After some time debugging we figured out the web connector sends out a "Expect:100 Continue" header and this is not supported by WebSEAL.
[https://www-01.ibm.com/support/docview.wss?uid=swg21626421

]1. Disabling the "Expect:100 Continue" functionality by putting setExpectedContinueEnabled to false in the "ThrottledFetcher.java" eventually solved the problem. The exact line can be found here:
https://github.com/apache/manifoldcf/blob/trunk/connectors/webcrawler/connector/src/main/java/org/apache/manifoldcf/crawler/connectors/webcrawler/ThrottledFetcher.java#L508

I'm not sure if this option is required for other environment, or that it can be disabled by default, or made configurable?

2. Another option would be to make the timeout configurable, as the WebSEAL docs state "The browser need to have some kind of timeout to to send the request body before exceeding intra-connection-timeout.". By default, the web connector request timeout exceeded the intra-connection-timeout of WebSEAL.

What is the best way to proceed and get a fixed for this in the web connector?

Kind regards,
Ferdi