Repository navigation
Make referrer-spoofing more granular #171
Description
Prerequisites
- I performed a cursory search of the issue tracker to avoid opening a duplicate issue
- Your issue may already be reported.
- I also searched the existing issues at https://github.com/gorhill/uMatrix/issues
- This is not a support issue or a question
- Support issues and questions are handled at /r/uMatrix
- I tried to reproduce the issue when...
- uMatrix is the only extension
- uMatrix with default lists/settings
- using a new, unmodified browser profile
- I am running the latest version of uMatrix
- I checked the documentation to understand that the issue I report is not a normal behavior
- I used the logger to rule out that the issue is caused by my ruleset
Description
I have referrer spoofing turned on globally.
referrer-spoof: * true
In the case of Trademe, I found I need to disable the spoofing for the login to work. In this very example, trademe.nz is the website, and the login process causes a XHR to api.trademe.co.nz. This request is denied unless the HTTP referrer specifies trademe.nz.
Without any changes to the ruleset, the XHR does not contain any HTTP referrer
field, which is weird, because I thought spoofing would have it send the
target domain's root document, i.e. https://api.trademe.co.nz, but there is
no header whatsoever. Consequently, the XHR yields a 401 response.
Since api.trademe.co.nz needs the referrer data for security decisions
(let's not discuss this…), I decided that I need to turn off referrer
spoofing, and naively thought I'd just add:
referrer-spoof: api.trademe.co.nz false
This, however, did not have any effect.
It turns out that instead I need to add
referrer-spoof: trademe.nz false
I.e. I need to turn off referrer-spoofing for all of trademe.nz, which also means that the site can now send referrer information to all the other 3rd party servers it contacts, even though I don't want that to happen.
While the rules syntax, i.e. specifying the rule, the context, and the verdict, makes sense in the case of e.g. web workers (I want to disable web workers in the context of trademe.nz and there is no 3rd party involved), referrer spoofing is different, as it involves a context, and a 3rd party.
In some ways, I think that a different rules syntax is required, i.e.:
* * referrer block
* api.trademe.co.nz referrer allow
this may be too much of an ask at this stage.
And even though I think the behaviour should be different, i.e. specifying referrer-spoof: trademe.nz true should cause trademe.nz to always see fake/spoofed referrer information, the current behaviour (trademe.nz is spoofed when used in a referrer header) should be better documented.
What do you think?
A specific URL where the issue occurs
https://trademe.nz login calling out to https://api.trademe.co.nz
Steps to Reproduce
- Disable referrer spoofing globally
- Enable referrer spoofing for
api.trademe.co.nz - Log in to https://trademe.nz and watch it fail
- Use network console to inspect the request and witness lack of HTTP referrer header
- Change referrer spoofing rule to
trademe.nzand watch the request toapi.trademe.co.nzreceive a spoofed referrer, which makes the login work.
Ruleset
referrer-spoof: * true
referrer-spoof: trademe.nz false
#referrer-spoof: api.trademe.co.nz false
Supporting evidence
Request with referrer-spoofing enabled, as well as disabled for api.trademe.co.nz:
POST https://api.trademe.co.nz/v1/authentication/token.json HTTP/1.1
Host: api.trademe.co.nz
[…]
Origin: https://trademe.nz
[…]
Request with referrer-spoofing disabled for trademe.nz:
POST https://api.trademe.co.nz/v1/authentication/token.json HTTP/1.1
Host: api.trademe.co.nz
[…]
Referer: https://trademe.nz/(modal:login)
Origin: https://trademe.nz
[…]
Your environment
- uMatrix version: 1.3.16
- Browser Name and version: Firefox 67.0.4
- Operating System and version: Debian sid
Activity
uBlock-user commentedon Jul 5, 2019
Username and password ?
uBlock-user commentedon Jul 5, 2019
Also FYI -- https://github.com/uBlockOrigin/uMatrix-issues/wiki/Per-scope-switches#spoof-referer-header
Why not login via
https://www.trademe.co.nz/Members/Login.aspx?url=%2fdefault.aspx?gorhill commentedon Jul 5, 2019
The hostname part in the referrer spoofing switch is matched against the context, aka the hostname of the root document, not the hostname of outgoing network requests. The context is the top left cell in the popup panel.
gorhill commentedon Jul 5, 2019
The reason the per-site switches are designed the way they are is simply because of the UI. To have a referrer-spoof switch which takes into consideration context and destination hostnames would require a new column in the matrix, which I decided was too much in the past. As a consequence per-site switches are somewhat blunt, and for a more precise control, the advice is to install a more specialezed extension for either referrer-spoofing, cookie control, etc.
madduck commentedon Jul 5, 2019
The mobile site is under the other URL.
madduck commentedon Jul 5, 2019
Yes, this is what I found as well. And I appreciate your explanation, aiming at simplicity. I guess what I am suggesting foremost is that the behaviour be properly documented, as https://github.com/uBlockOrigin/uMatrix-issues/wiki/Per-scope-switches#spoof-referer-header doesn't make this obvious.
I personally wouldn't think another column would be too much, and it's not like it'll open a can of worms, because I can't think of any other columns that would then also need to be added.
Adding such a column would give me the freedom to really express what I want and expect to happen.
I, for one, find it rather weird that in order for a website to be named as referrer in a specific 3rd party request, I need to whitelist the website itself, thereby allowing it to be named as referrer in all outgoing requests. Generally, I want to inhibit someone receiving information, not someone sending it, and then I want to punch specific holes through this ruleset. In the case of the referrer, some sites rely on it. I want to whitelist those sites, not all sites that refer to the arguably broken ones relying on such data.
gorhill commentedon Jul 5, 2019
I will consider adding
referreras a new type (for matrix-filtering purpose), but adding a column to the matrix is too much of a big change for now, so whoever wants finer control of referrer spoofing can always manually craft them.uBlock-user commentedon Jul 5, 2019
So what would be the rules be like ?
gorhill commentedon Jul 5, 2019
Like any matrix filtering rule:
source destination type action, exampleuBlock-user commentedon Jul 5, 2019
Would
* * referrer blockwork ?gorhill commentedon Jul 5, 2019
Sure, matrix-filtering rules are all handled with the same core algorithm.
uBlock-user commentedon Jul 5, 2019
and would we able to configure it from the uMatrix's popup panel via the referrer switch ?
gorhill commentedon Jul 5, 2019
The switch would just be converted to a matrix-filtering rule from a per-site switch rule.
uBlock-user commentedon Jul 5, 2019
No I meant if I have to disable referrrer spoofing for an embedded domain as an iframe on some website, would I need to do from the My rules tab manually or will I able to do it via the popup panel itself ?
gorhill commentedon Jul 5, 2019
All matrix rules use the hostname of the root context as
sourceand always have, never that of an embedded document (iframe), so the answer is you will be able to do it through the popup panel.[-]Please clarify referrer spoofing, or new rule syntax?[/-][+]Make referrer-spoofing more granular[/+]madduck commentedon Jul 5, 2019