openshift.org

This is where you should report issues arising from the subscription filters.
Locked
mwringe
New Member
New Member
Posts: 8
Joined: Thu Mar 09, 2017 8:26 pm

openshift.org

Post by mwringe »

OpenShift openshift.org uses Hawkular Metrics hawkular.org to store its metrics about container usage (eg CPU, memory, network, etc)

We are running into issues where if a user has an adblocker installed, our console page for graphing container usage fails.
This is because the url to Hawkular Metrics can contain /metrics/metrics in it.

Eg a sample url may look like the followiing:

Code: Select all

https://hawkular-metrics.example.com/hawkular/metrics/metrics/stats/query
There are no ads on this page and this url is only being used to fetch metrics from our Hawkular Metrics instances so that they can be graphed.

What can we do we to get our urls to not be blocked anymore?
User avatar
smed79
Liste AR/FR Author
Liste AR/FR Author
Posts: 15839
Joined: Sun Jan 17, 2010 4:00 am
Location: EasyList Forum

Post by smed79 »

mwringe wrote:this url is only being used to fetch metrics from our Hawkular Metrics instances so that they can be graphed
This filter is part of EasyPrivacy which is an optional subscription (manually added by user) deputed to block tracking services.
•► Read RULES / Use forum Search
••► Don't post clickable links
•••►Upload screenshots at imgbb.com
mwringe
New Member
New Member
Posts: 8
Joined: Thu Mar 09, 2017 8:26 pm

Post by mwringe »

Yes, its part of EasyPrivacy, which is indicated in uBlockOrigin and links to this forum. Sorry, is this not the right location to ask?

This is not a tracking service and is being incorrectly flagged as invalid just because it has "/metrics/metrics" in its name.
User avatar
smed79
Liste AR/FR Author
Liste AR/FR Author
Posts: 15839
Joined: Sun Jan 17, 2010 4:00 am
Location: EasyList Forum

Post by smed79 »

mwringe wrote:This is not a tracking service...
Provide a real testable online example... and wait an easylist author see this topic.
•► Read RULES / Use forum Search
••► Don't post clickable links
•••►Upload screenshots at imgbb.com
mwringe
New Member
New Member
Posts: 8
Joined: Thu Mar 09, 2017 8:26 pm

Post by mwringe »

For the OpenShift issues, its not going to be that easy right now. Its in a management console and not something which would be publicly available. Our online version is in developer preview so we might be able to see if we can get someone access to test (https://console.preview.openshift.com).

Otherwise you could of course install OpenShift Origin locally and verify with that, but I somehow think that is not what you are looking for :)

Its a bit strange to block anything if it has "/metrics/metrics" in it as that could cover a lot of different possible URL patterns.

I can provide you documentation if that would be helpful at all:

Our bugzilla for this issue: https://bugzilla.redhat.com/show_bug.cgi?id=1430888

Documentation for how to fetch the metrics manually which shows the "/metrics/metrics" URL in a lot of our examples: https://github.com/openshift/origin-met ... -a-project

Is there anything else we can provide which would help to get this looked at? I realise that not having an good quick site which can be used is a bit of a pain. If you do need access to something live, I would have to figure out how to get that setup for you. Would you need an actual site or just something with Hawkular Metrics running that you could curl into?
User avatar
fanboy
EasyList Author
EasyList Author
Posts: 12223
Joined: Wed Sep 05, 2007 8:17 pm

Post by fanboy »

Given the difficulty of fixing this, can openshift just change the folder name?
mwringe
New Member
New Member
Posts: 8
Joined: Thu Mar 09, 2017 8:26 pm

Post by mwringe »

How is it difficult to fix this exactly? The privacy blockers appear to be too strict with regards to blocking everything with "/metrics/metrics" in it.

We cannot easily change this in OpenShift without breaking other existing REST based clients. We are looking into other alternatives, but its a bit crazy that we have to rearrange our paths just because a privacy blocker is incorrectly flagging things.
User avatar
fanboy
EasyList Author
EasyList Author
Posts: 12223
Joined: Wed Sep 05, 2007 8:17 pm

Post by fanboy »

I'm open to filter suggestions.
wgordon
New Member
New Member
Posts: 2
Joined: Wed Mar 15, 2017 1:45 pm

Post by wgordon »

Since this URL is an API endpoint, we already have customers who have built solutions around this format. Since at the very least, this is affecting paying customers, would it be possible to provide a whitelist/exclusion as part of EasyList for https://metrics.*.openshift.com? This at least provides a very limited scope of what is being whitelisted/excluded, and the above format will be the same for at least one sub-section of our customers.

Out of curiosity, what is it about the "/metrics/metrics" format that you've seen have ads on it...or is it more for blocking tracking scripts?
mwringe
New Member
New Member
Posts: 8
Joined: Thu Mar 09, 2017 8:26 pm

Post by mwringe »

Fixing it for https://metrics.*.openshift.com wont help that much since it will still fail on all the other installations which are not using that url.

An exclusion for "hawkular/metrics/metrics" would work better, as this should work across all installations of OpenShift
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

mwringe wrote: Thu Mar 16, 2017 12:58 pm An exclusion for "hawkular/metrics/metrics" would work better, as this should work across all installations of OpenShift
That's a pretty broad exception and has high potential to be abused. Say if a "bad" site is getting blocked on metrics/metrics, if we used your suggestion, all they'd have to do is add hawkular to their path and problem solved. Do you see where I'm going with this?

Same thing i said here: viewtopic.php?f=64&t=35937 cases like this are almost impossible to fix on a filter level.

I'm not an EasyList author so final decision isn't mine. Everything I said IMO of course. :mrgreen:
"If it ain't broke don't fix it."
wgordon
New Member
New Member
Posts: 2
Joined: Wed Mar 15, 2017 1:45 pm

Post by wgordon »

@mwringe I agree that it doesn't solve the problem for all installations, but the *.openshift.com whitelist would at least fix it for Red Hat hosted installations, where we might be expected to be able to provide assistance.

Perchance could we get an author's opinion of both of these options?
mwringe
New Member
New Member
Posts: 8
Joined: Thu Mar 09, 2017 8:26 pm

Post by mwringe »

Lanik wrote: Thu Mar 16, 2017 3:38 pm
mwringe wrote: Thu Mar 16, 2017 12:58 pm An exclusion for "hawkular/metrics/metrics" would work better, as this should work across all installations of OpenShift
That's a pretty broad exception and has high potential to be abused. Say if a "bad" site is getting blocked on metrics/metrics, if we used your suggestion, all they'd have to do is add hawkular to their path and problem solved. Do you see where I'm going with this?
Its also a pretty broad assumption relying just on "/metrics/metrics" being in the path anywhere.

I fail to see the argument here, if they can change the URL path, then they can easily just rename it as something like "metric/metrics" or "m/metrics" or "m/m"

Any of those will get around this arbitrary URL path that is being used.

We are even considering doing the same thing to get around this issue. Just rename any part of this and we get through the filter. We would prefer not to do this as it beaks compatibility with some of our clients. But it looks like we might not be able to make any progress to get this resolved at the filter level.

I don't understand how some "bad" site somewhere at sometime used "/metrics/metrics" in their tracking, so now we are trying to block anyone on the internet from ever having that in their URL path. This makes no sense to me.
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

wgordon wrote: Tue Mar 28, 2017 2:12 pm Perchance could we get an author's opinion of both of these options?
One of the authors has already voiced their opinion here: https://forums.lanik.us/viewtopic.php?f=64&t=35937 I don't imagine that's going to change since these issues are so similar.
mwringe wrote: Tue Mar 28, 2017 6:57 pm I fail to see the argument here, if they can change the URL path, then they can easily just rename it as something like "metric/metrics" or "m/metrics" or "m/m"
The idea is to run their scripts under a white list so they'll change their site/scripts etc to take advantage of an allowed metric/metrics filter.
mwringe wrote: Tue Mar 28, 2017 6:57 pm I don't understand how some "bad" site somewhere at sometime used "/metrics/metrics" in their tracking, so now we are trying to block anyone on the internet from ever having that in their URL path. This makes no sense to me.
Maybe I'm not explaining it correctly or whatever, but if you need examples of such abuse see this: https://forums.lanik.us/viewtopic.php?f=62&t=34172
"If it ain't broke don't fix it."
mwringe
New Member
New Member
Posts: 8
Joined: Thu Mar 09, 2017 8:26 pm

Post by mwringe »

Lanik wrote: Wed Mar 29, 2017 12:18 am
wgordon wrote: Tue Mar 28, 2017 2:12 pm Perchance could we get an author's opinion of both of these options?
One of the authors has already voiced their opinion here: https://forums.lanik.us/viewtopic.php?f=64&t=35937 I don't imagine that's going to change since these issues are so similar.
Which in unfortunate. Essentially the blocking filters are being too broad in their approach and they are having collateral damage. And for whatever reason, the people maintaining these lists don't seem to be able to make sure that their filters are working as expected.

I think the proper fix here is to figure out what is causing those filters to exist in the first place, and to tighten the restriction so that it only blocks what it is mean to block.

I wouldn't even be surprised if what originally caused these filters to be applied in the first place hasn't already been updated to get around the filter.
Lanik wrote: Wed Mar 29, 2017 12:18 am
mwringe wrote: Tue Mar 28, 2017 6:57 pm I fail to see the argument here, if they can change the URL path, then they can easily just rename it as something like "metric/metrics" or "m/metrics" or "m/m"
The idea is to run their scripts under a white list so they'll change their site/scripts etc to take advantage of an allowed metric/metrics filter.
mwringe wrote: Tue Mar 28, 2017 6:57 pm I don't understand how some "bad" site somewhere at sometime used "/metrics/metrics" in their tracking, so now we are trying to block anyone on the internet from ever having that in their URL path. This makes no sense to me.
Maybe I'm not explaining it correctly or whatever, but if you need examples of such abuse see this: https://forums.lanik.us/viewtopic.php?f=62&t=34172
I think you are making a case for why whitelisting things is bad (which I don't necessarily disagree with, although what you linked to sounds more like a bug than anything else)

There may have been some confusion over how I wrote things. When I said 'exclusion' for "/hawkular/metrics/metrics" I didn't mean to just whitelist anything that matches that. I meant to update the filter so that "/hawkuar/metrics/metrics" wouldn't necessarily trigger the overly broad "/metrics/metrics" check.

The better option would be to fix the "/metrics/metrics" exclusion so that its not so broad, but it doesn't look like that will be possible.

I still stand by my argument:
mwringe wrote: Tue Mar 28, 2017 6:57 pm I don't understand how some "bad" site somewhere at sometime used "/metrics/metrics" in their tracking, so now we are trying to block anyone on the internet from ever having that in their URL path. This makes no sense to me.
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

mwringe wrote: Wed Mar 29, 2017 9:30 pm There may have been some confusion over how I wrote things. When I said 'exclusion' for "/hawkular/metrics/metrics" I didn't mean to just whitelist anything that matches that. I meant to update the filter so that "/hawkuar/metrics/metrics" wouldn't necessarily trigger the overly broad "/metrics/metrics" check.

The better option would be to fix the "/metrics/metrics" exclusion so that its not so broad, but it doesn't look like that will be possible.
I'm not disagreeing with any of that however based on the ABP Syntax or uBO Syntax I don't see any way of allowing /hawkuar/metrics/metrics without opening it up to abuse. If you're seeing something we're not I'm also open to suggestions.
mwringe wrote: Wed Mar 29, 2017 9:30 pm I still stand by my argument:
mwringe wrote: Tue Mar 28, 2017 6:57 pm I don't understand how some "bad" site somewhere at sometime used "/metrics/metrics" in their tracking, so now we are trying to block anyone on the internet from ever having that in their URL path. This makes no sense to me.
It's your right to stand by your argument. I just wouldn't get my hopes up of fixing the so called "bug". :mrgreen:
"If it ain't broke don't fix it."
mwringe
New Member
New Member
Posts: 8
Joined: Thu Mar 09, 2017 8:26 pm

Post by mwringe »

Lanik wrote: Thu Mar 30, 2017 1:00 am
mwringe wrote: Wed Mar 29, 2017 9:30 pm The better option would be to fix the "/metrics/metrics" exclusion so that its not so broad, but it doesn't look like that will be possible.
I'm not disagreeing with any of that however based on the ABP Syntax or uBO Syntax I don't see any way of allowing /hawkuar/metrics/metrics without opening it up to abuse. If you're seeing something we're not I'm also open to suggestions.
I looks like the filtering is quite limited in comparison to something like regular expressions.

But adding an exception for hawkular/metrics/metrics was only suggested as an option to get around all of this.

I still think the real solution here is to figure out why you are using such a broad filter for "/metrics/metrics" and update the filter to be more specific to whatever caused caused this filter to be added in the first place. That should be possible with the limited filtering options you have available to you, assuming there is something else in the path which could be used to help narrow it down.

What originally caused this to be added? What exactly are you trying to block with this? Can we not see what they are using and update the filter to be more specific so that it only blocks what its meant to block?
Lanik wrote: Thu Mar 30, 2017 1:00 am
mwringe wrote: Wed Mar 29, 2017 9:30 pm I still stand by my argument:
mwringe wrote: Tue Mar 28, 2017 6:57 pm I don't understand how some "bad" site somewhere at sometime used "/metrics/metrics" in their tracking, so now we are trying to block anyone on the internet from ever having that in their URL path. This makes no sense to me.
It's your right to stand by your argument. I just wouldn't get my hopes up of fixing the so called "bug". :mrgreen:
I think most people would agree that having such a broad filter and blocking sites its not meant to block would be considered a bug
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

mwringe wrote: Thu Mar 30, 2017 4:02 pm I looks like the filtering is quite limited in comparison to something like regular expressions.
There's also this and yes ABP Syntax supports regex. Like I said I'm all ears if you have suggestions.
mwringe wrote: Thu Mar 30, 2017 4:02 pm What originally caused this to be added? What exactly are you trying to block with this? Can we not see what they are using and update the filter to be more specific so that it only blocks what its meant to block?
Only a filter author would know this provided there's a "paper" trail.
mwringe wrote: Thu Mar 30, 2017 4:02 pm I think most people would agree that having such a broad filter and blocking sites its not meant to block would be considered a bug
I seriously doubt it but you're entitled to your opinion. :P
"If it ain't broke don't fix it."
GoGoris
New Member
New Member
Posts: 1
Joined: Fri Nov 10, 2017 8:17 am

Post by GoGoris »

I am having the same problem and I don't see how hawkular*/metrics/metrics could be abused but /metrics/metrics not.
A malicious site could just as well change /metrics/metrics as add hawkular in their url...

Can't you make it as specific as possible and unblock https://hawkular-metrics.*/hawkular/metrics/metrics/* ?
User avatar
LanikSJ
Site Owner
Site Owner
Posts: 1806
Joined: Thu Feb 15, 2007 7:44 am
Location: /dev/null

Post by LanikSJ »

See below thread these types of issues won't go anywhere. There's been no traction in 6 months.

viewtopic.php?f=64&t=35937

If one of the EL authors changes their mind they can reopen this thread.
"If it ain't broke don't fix it."
Locked