[COUCHDB-3257] Replicator accepts and then returns invalid urls

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Component/s: Database Core, Replication
• Assignee: Unassigned
• Reporter: Robert Kowalski
• Created: 16/Dec/16 12:11
• Updated: 15/Nov/17 12:45
• Resolved: 15/Nov/17 12:45

Description

We have an issue that manifests for us in Fauxton but will manifest in any other web browser / url parser.

The replicator accepts invalid urls. This means it will also return invalid urls on request. These url make standard-conforming url parsers bail. Example:

https://rocko:pass#[e-mail]/blerg is not valid url syntax. The hash has to be encoded.

Discussion from #whatwg:

12:17:03 < robertkowalski> annevk: question to the url spec
12:17:16 < robertkowalski> before i open an issue / invetsigate further
12:18:11 < robertkowalski> new URL('https://rocko:pass#[e-mail]/blerg')
12:18:16 -!-frivoal [~frivoal@2400:2650:86c0:a500:6c4e:56ad:30ff:8140] has joined #whatwg
12:18:18 < robertkowalski> throws because of the hash
12:19:29 < robertkowalski> i haven't found a section regarding passwords and special / reserved chars. is this a bug in the spec? it limits the amount
                           possible passwords a lot
12:20:06 < nox> robertkowalski: It should be encoded.
12:20:23 < annevk> Yeah, you can encode it
12:21:16 < annevk> robertkowalski: the specification basically doesn't want you to use URLs to encode username/password
12:21:29 < annevk> robertkowalski: https://url.spec.whatwg.org/#url-syntax doesn't allow them
12:21:46 < annevk> robertkowalski: (see note at the end of that section)
12:22:39 < annevk> robertkowalski: the reason that throws though I think is because # is seen as the start of the path and then a host cannot contain :
12:22:43 -!-frivoal [~frivoal@2400:2650:86c0:a500:6c4e:56ad:30ff:8140] has quit [Ping timeout: 258 seconds]
12:22:51 < annevk> robertkowalski: well, because :pass is not a valid port
12:23:54 < annevk> robertkowalski: for that, see how https://url.spec.whatwg.org/#authority-state and also the host state will treat # as the end of that
12:24:25 < annevk> robertkowalski: and https://url.spec.whatwg.org/#port-state for how port will return failure for non-digits
12:27:16 < robertkowalski> thank you
12:27:46 < robertkowalski> the replciator in couchdb accepts urls with hash as part of the password
12:27:57 < robertkowalski> and when we pull them out and want to use them in the browser
12:27:59 < robertkowalski> it explodes
12:30:14 < nox> robertkowalski: new URL('https://rocko:[e-mail]/blerg')
12:31:18 < robertkowalski> ty nox - we run into a chicken egg problem here. as we use `new URL` to parse the URL ^^
12:31:31 < robertkowalski> so we probably have to fix that in the couch api, not in the frontend
12:36:15 < annevk> robertkowalski: yeah, it sounds like the Couch DB API parses URLs differently from browsers
12:36:43 < annevk> robertkowalski: that will cause subtle bugs

Proposal:

• Reject invalid urls and add a automatic migration strategy for invalid urls in the replicator

Issue Links

• links to: GitHub Pull Request #824