Ending support for JSONP
We've all played with JSON a bit. It usually looks like this (the spacing doesn't matter, but it's most likely to be seen in 1 like below, or carefully indented like in site configs):
{ "Item1": "lol", "Item2": "jk", "Item3": "nm" }
There are security restrictions in browsers (we call it CORS - Cross-Origin Resource Sharing) to prevent them from working with data from other domains directly. We need to be really, really sure we should be reaching out across domains to share data or else you can do a lot of nefarious stuff.
JSONP (JSON with Padding) lets us sidestep this restriction. Boiled down, it basically lets you send JSON that's formatted like a Javascript function. The above example, redone as JSONP (with a callback of 'test'):
test({ "Item1": "lol", "Item2": "jk", "Item3": "nm" });
Literally just embeds the JSON inside test();, but that's really powerful. It can enable all sorts of neato integrations across sites, like pulling comment counters into a blog page or user data into a navigation bar.
This was state-of-the-art a few years back. Now, we don't really need or use it much, and it's generally regarded as a security risk to allow it.
The good news is it's off by default in Vanilla and only allowed (whitelisted) in a few key places where we're 100% sure we're not leaking sensitive data (basically 2 spots, just for the WordPress plugin).
The bad news is that there's an old config setting to turn it on globally, and a dozen or so customers have it turned on. We don't know why, but we need to get it out of there.
The config setting is Garden.AllowJSONP = true and it's getting expunged from all configs in the next patch release.
These sites (usually both staging + production) have JSONP enabled:
- unknownworlds
- kixeye
- collegeconfidential
- thebump
- stanwinstonschool
- edmunds
- aat
- msa
- oculus
- battlefront-ea
- ohmconnect
- atkins
- pqsystems
- valify
- yd
- dentistry
Your mission, CSMs & Support: Reach out and find out if they're actually using JSONP in a site integration. If they are, get exactly what data they're pulling down so we can help support them in a more selective way than globally allowing JSONP.
Comments
-
Mine:
- unknownworlds DONE
- oculus DONE
- battlefront-ea DONE
- ohmconnect DONE
- dentistry DONE
- yd (asked for recommendations in their sso task where we turned this on)
Here is the generic email I did if you want to steal :)
0 -
@ValR AAT has been reached out too. Thank you @shaunamcclemens for the email template :)
1 -
If we deploy this next week and something does break, note you can temporarily re-add that config setting to "fix" it while we sort things out and determine the real solution. We're simply removing the config from all sites, not fundamentally killing it from core (yet).
0 -
AAT has confirmed they do not use JSON at all
0 -
To clarify, in case there's any confusion: this is about limiting Vanilla serving JSONP. Addons or components of Vanilla can still consume JSONP from an external source. For example, I think there's still some support in jsConnect for an authentication provider to respond with JSONP. This should not be affected by the update discussed here, because the JSONP is served by an external source (the authentication provider).
0 -
Here's Stan Winston School:
We are currently using JSONP to pull contest entrants and display contest entries each month when we hold a contest on our forums.
Here are 2 examples:
1: Contest Entrants
Because Vanilla Forums has no way to message users individually as they post to a category, and no tool to message everyone who has posted to a category retroactively, I have developed the following tool to list out every unique user that has posted to a specific contest category and generate a link directly to their message screen so I can past a message to each of them. While this helps, it still means I have to manually paste and send hundreds of messages.
http://www.vexfx.com/sws/Halloween2018/entrants/
2: Contest Judging
Because there is no fast way to browse posts I have created the following tool to allow quick previewing of all posts within a category, sorted by poster. That way we can review that each individual has no more than 3 entries, and we can then scan all entries for judging. Use UP/DOWN arrow keys to quickly cycle through posts after the data has loaded.
http://www.vexfx.com/sws/Halloween2018/judging/
With JSONP suddenly going away, what is the alternative you have lined up to replace it?
I’ve been hoping for a real Vanilla Forums API for years, and hope some progress has been made there.
I translated earlier that this feature was going away, so I can let Chris know that I can flip the config back on for him when the change happens. Is there some other direction I should be pointing him in?
0 -
I translated earlier that this feature was going away, so I can let Chris know that I can flip the config back on for him when the change happens.
I'm pretty sure it'll be un-flipped the next time a deploy/"utility update" happens.
Is there some other direction I should be pointing him in?
Trusted domains. JSONP lets users dodge CORS protections, which we don't want to encourage. Properly configuring trusted domains should allow users to work within CORS. Sites on trusted domains will be allowed to pull data from a Vanilla site in regular JSON. There's no need for the JSONP workaround.
0 -
I see, I've let Chris know and will pop back up if there are any concerns. Thanks Ryan!
0 -
Thanks @ValR and @shaunamcclemens
I've reached out to Valify. It's done.
0 -
@Linc From Spark People (aka devs responsible for Akins)
We use the JSONP stuff in two ways.
We call https://community.atkins.com/api/v1/discussions.json and https://community.atkins.com/api/v1/discussions/bookmarked.json to grab the related posts to populate a section of the main dashboard page related to message board posts.
We also call https://community.atkins.com/api/v1/profile/notifications.json to figure out how many unread messages the user has to populate a little alarm icon in the header.
We do have the SSO to you that we run people through when they sign into the platform site (it’s there mainly to allow us to grab the info above). We don’t think that it involves any jsonp, but I’m honestly not sure.
0 -
EA Battlefront - removed and everyone lived. All good.
Unknown Worlds:
Hello Shauna,
yeah, we still actively use it on two blogs:
https://unknownworlds.com/subnautica/
https://unknownworlds.com/ns2/
We use it to pull user data from forums (https://forums.unknownworlds.com/profile.json?callback=jQueryxxx).
Have a great week!
I'm going to ask him what kind of data to see if we can maybe pull it over API.
0 -
I'm going to ask him what kind of data to see if we can maybe pull it over API.
That's a fine pursuit, but in the meantime, they can probably just use trusted domains. They'd have to setup unknownworlds.com as a trusted domain in their site dashboard. Once done, they should be able to pull regular ole JSON from the same URL, but without the JSONP
callbackparameter/workaround.0 -
@Ryan any suggestions for the below
Site: https://console.vanilladev.com/sites/view/overview/5020761/stanwinstonschool.vanillacommunity.com
Chris writes:
Access to XMLHttpRequest at 'http://stanwinstonschool.vanillaforums.com/api/v1/discussions/list.json?access_token=********&page=1-1000&_=1543353623550' from origin 'http://www.vexfx.com' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource.
I tried:
stanwinstonschool.com
vexfx.com
and
*stanwinstonschool.com
*vexfx.com
and
*.stanwinstonschool.com
*.vexfx.com
and
*.stanwinstonschool.com*
*.vexfx.com*
and
*.stanwinstonschool.com/*
*.vexfx.com/*
Just to see if different wildcard placement has an impact.
Wait. I think this is just an https problem.
0 -
We discussed this in the support sprint room, but to recap:
It looks like the client's issue is due to him using the wrong domain. He should be using forums.stanwinstonschool.com, but he's using stanwinstonschool.vanillaforums.com. Since he's using the vanillaforums.com domain, the request is being redirected to forums.stanwinstonschool.com and the CORS headers are lost. Avoiding the redirect by using the proper domain should result in the CORS headers making it to his browser.
I tested this out, myself, making calls from vexfx.com, and was able to receive the
Access-Control-Allow-Originheader.0 -
From College Confidential:
With the new dev team, it's going to take some time for them to figure out what we have and where. What's the deadline on this?
0 -
We're turning it off in the release next week.
You could turn it back on again afterward via the config, but it's a non-trivial security issue since it can leak personal data.
0 -
Unknown worlds came back wth this:
hmm, that sounds like a really good solution. I tried it, but without luck. I added my domains at https://forums.unknownworlds.com/dashboard/settings/security - but the query (https://forums.unknownworlds.com/profile.json) returns 404s for when I switch from JSONP to JSON.
Here's the code I'm using:
$.ajax({ type: "GET", dataType: "json", url: "https://forums.unknownworlds.com/profile.json" })0 -
Are they signed into the forum? If you aren't signed in, you'll get a 404 "User not found" when trying to access /profile (and, by extension, /profile.json). This is true for JSONP too (e.g. /profile.json?callback=callback_name).
1 -
Btw jQuery automatically add
?callback={CALLBACK}to the request. No need to add it to the URL.0 -
After digging into this a little more, I think this is something we'll need to address on the Vanilla side. Vanilla is currently telling the browser that it's okay to consume data from the site, but not that it's okay to send the user's credentials (cookies).
I've created a PR to (hopefully) address this: vanilla/vanilla#8088
Once this patch is in, and deployed, the client will most likely need to update their AJAX request to something like:
jQuery.ajax({ type: "GET", dataType: "json", url: "https://forums.unknownworlds.com/profile.json", crossDomain: true, xhrFields: { withCredentials: true }, })The key change here is
withCredentials: true, which configures the jQuery request to send the user's credentials (cookies) along with the request.0 -
@shaunamcclemens could you please forwardne that email?
0 -
(just noting I've updated unknown worlds)
0 -
Sidenote: we should update these docs?
https://docs.vanillaforums.com/help/sso/jsconnect-quickstart/#jsonp
0 -
@shaunamcclemens The CORS "credentials" update was deployed to public clusters. Unknown Worlds should give it another shot. I did some basic testing and it seemed to function as expected.
0 -
No, that is in relation to jsConnect, which still uses JSONP. (The endpoint on THEIR server responds with JSONP, not Vanilla itself.) It's also purely conceptual, not implying it's to be used while coding against Vanilla.
I think there is some other section about JSONP in the docs that needs to be struck, but it's eluding me today.
0
