[COUCHDB-3003] Non-ASCII strings in headers should be URL encoded

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Won't Fix
• Component/s: HTTP Interface
• Assignee: Unassigned
• Reporter: Dirkjan
• Created: 26/Apr/16 19:11
• Updated: 02/Sep/18 07:06
• Resolved: 02/Sep/18 07:06

Description

From this CouchDB-Python issue:

https://github.com/djc/couchdb-python/issues/281

And this requests issue:

https://github.com/kennethreitz/requests/issues/3098

There is this comment:

The header parsing is done by httplib, in the Python standard library; that is the part that failed to parse. The failure to parse is understandable though: servers should not be shoving arbitrary bytes into headers.

Using UTF-8 for your headers is extremely unwise, as discussed by RFC 7230:

Historically, HTTP has allowed field content with text in the ISO-8859-1 charset [ISO-8859-1], supporting other charsets only through use of [RFC2047] encoding. In practice, most HTTP header field values use only a subset of the US-ASCII charset [USASCII].

In this instance it's not really possible for us to resolve the problem. The server should instead be sending urlencoded URLs, or RFC 2047-encoded header fields. Either way, httplib is getting confused here, and we can't really step in and stop it.