MSE's SEO concerns
One of our newest Enterprise clients has a lot of SEO concerns, and I’d like to see if I can get some feedback here before opening a ton of estimates for this client.
It is worth nothing that this client will be paying 17k per month.
Sitemaps
Would the sitemap be configurable to only show organically indexable URLs? I.e. not blocked by robots, self-referencing canonicals etc.?
404s
404 page to return 404 status and be served directly and not via a redirect
404 page to offer users/bots appropriate internal linking/route back into site
Is this the case already? If not, we would consider custom work on this?
503s
1. Return 503 status and be served directly rather than going through redirect
2. Configurable retry-after value
3. Page to offer suitable/appropriate messaging
This is a default page in Vanilla, I am not confident that we’d do custom work to this page, but am not too sure about 1&2, any input would be helpful!
Canonicalisation of forum homepage
Ensure forum homepage has self-referencing canonical tag
We've noticed on tomtom and digital spy forums that the canonical tag on the forum homepage actually points to the category page. In the case of tomtom this means that an organic search for 'tomtom forum' returns the main category page rather than the forum homepage
I thought I had notes on this, but can't seem to find them :S
Comments
-
Sitemaps are not customizable, although I know @Todd was looking at going a different way.
The 404 is a standard 404- when I did a header check it replies with the appropriate response. The 404 page still has the header nave, so the bots are not stuck on a blank page
The 503 page I believe is not configurable
Canonical on the homepage, I believe @patrick_kelly did work on that for a custom plugin.
2 -
i know you're out right now, so this is for when you get back!!!!
We mean the following link rel="canonical" tag - can see it present on prod but it's incorrect -https://mse.vanillacommunities.com/discussion/6/what-would-you-like-to-see-in-the-new-forum would expect the canonical to be <link rel="canonical" href="https://mse.vanillacommunities.com/discussion/6/what-would-you-like-to-see-in-the-new-forum" /> but its <link rel="canonical" href="https://mse.vanillacommunities.com/categories/general" /> --
any advise on these?
0 -
There are a few things here that might be product feature requests, sponsored development, or possibly custom work:
maybe @Todd or @Linc has opinions here, before i create these as estimates ;)
1--- Breadcrumb structured data
Current implementation uses data-vocabulary.org which is outdated.
Can we please update this to use schema.org/BreadcrumbList. Ideally this would be implemented using JSON-LD but microdata would be acceptable as a fallback option
2--- Forum post structured data
Current implementation returns multiple errors via Google's structured data testing tool:
Can this be updated to include the three additional required fields; datePublished, image and publisher
3--- not enough 404s
It seems like the discussion page content is entirely determined by the unique identifying number. For instance, this is a discussion URL - https://mse.vanillacommunities.com/discussion/4/how-to-start-a-great-discussion. if you remove everything after the identifying number then the page is still returned - https://mse.vanillacommunities.com/discussion/4/
Also, if you just change the number then it will change the content - e.g. https://mse.vanillacommunities.com/discussion/3/how-to-start-a-great-discussion
This also relates to #6 (404 config) as it is impossible to trigger a 404 response with incorrect URLs on discussions - e.g. https://mse.vanillacommunities.com/discussion/3/this-will-not-trigger-a-404-response
We've noticed this is also an issue on digital spy - e.g. https://forums.digitalspy.com/discussion/2348659/sldkjalksdjlkwsjdlksjdlkajs
Entire URL structure should define the content that is returned and not just the unique identifying number. Any incorrect URLs should return a 404 response status
0 -
Is there any evidence of our default 503s being bad for SEO?
I'm pretty sure we had already discussed our 503 capabilities in past discussions...
1 -
1--- Breadcrumb structured data
This is already implemented for knowledge base in JSON-LD. We could probably bring that over to forum pretty easily. (BreadcrumbModel is already implemented for forum discussions).
3--- not enough 404s
We use canonical URLs. Google follows these. Serving 404s for non-canonical versions of a URL is bad for SEO, and also breaks existing backlinks that might exist to a discussion.
Example when visiting this discussion without a the slug at the end:
This is considered best practice, both for SEO, and for user experience.
1 -
per your example it’s working properly. The only issue I know is on the homepage using the improper canonical. Remember the point of Canonical is only to offer a suggestion to Google when multiple paths ( think about when we add #latest to a link), for what the preferred page is. It’s not really SEO as much as crawling optimization.
1 -
I had worked on a different sitemap structure and it is sitting in a PR right now. We can work on getting that merged and try it out on something like our success community. It puts the new method behind a feature flag due to its risk of impacting SEO.
1 -
First of all, thank you all for the help and insightful comments. I will try and put together some CSM-docs from this conversation after the dust has settled for MSE.
Now, onto my new question from MSE:
question
1. Analysis of default Vanilla set-up suggests render blocking issues, how do we address these?
requirements
1. Adhere to Google's best practice guidelines for page speed performance
2. All page templates pass Google PageSpeed Insights test (Pass score TBD)
notes
Testing via Google's PageSpeed Insights tool highlights a number of opportunities to optimise performance. These mainly centre aroind addressing render-blocking resources and preloading key requests. Here is a link to a sample test:
0 -
We already have a feature flag for render blocking issues, I’d really like to get it out as the new default.
In any case I can dig it up. It’s in one of the previous release notes though and we use it on all of our own sites.
1 -
Thanks Adam! (for anyone interested its this: https://success.vanillaforums.com/kb/articles/76-2019-003#seo-performance)
0 -
notes
Testing via Google's PageSpeed Insights tool highlights a number of opportunities to optimise performance. These mainly centre aroind addressing render-blocking resources and preloading key requests. Here is a link to a sample test:
that's surprising, I ran Acer and a few other sites before and they scored a lot higher. Did we change some stuff?
Any particular reason why this wouldn't be enabled by default? I'm thinking Golfworkx and hearst would be keen to get more SEO and increase PVs
0 -
I'm working on a user story to start enabling it by default.
The biggest reason we haven't flipped the switch is because it can break existing pockets if they use jquery. We need some UI when enabling this to inform our site admins that they need to update their pockets and why. I'm making an issue this sprint, so I think we'll be on track for a toggle in the dashboard and enabling by default in Vanilla in the next release or 2.
0 -
MSE pushed back a little bit on this one:
Best pratcice on canonicals is applicable when it's a genuine variant (e.g. parameters etc.). However, a URL like this https://mse.vanillastaging.com/discussion/19/sdfkjhfkjhdfkjshdfkjhsdkjfhk is clearly not a genuine variant of the page and should 404.
If it's not possible to address this for launch then we'll have to accept the risk and monitor for crawl errors. We will re-raise if it becomes an issue
So , sounds like it won't be a launch blocker, but if anyone has any comments on this, let me know!
0 -
@Adam Charron mse is right we should not serve a canonical on a 404
0 -
I’m not sure. I read as canonical on 404 is wrong.
and I think they are asking for a static 404 page to serve rather then allowing 404 wrong slugs to live.
not sure best practice on the second one as much as a preference
1 -
Our 404 pages canonicalize to /home/file-not-found. I don't think that's what they were getting at though.
0 -
I believe the issue is that
if i go to https://staff.vanillaforums.com/adjaksdfjskla
I get a page not found
if i go to https://staff.vanillaforums.com/discussion/ajfdkasjfklasjfklajlk
i get a page not found
if i go to https://staff.vanillaforums.com/discussion/889/ajfdkasjfklasjfklajlk i still go to this discussion
i believe that this is in case the discussion title changes, it makes sense to still land on discussion # 889, regardless of what the title is now and that the SEO people are not thinking about forum functions as much as SEO things.
0 -
@shaunamcclemens Then they must not be good SEO people. If someone had shared a previous link to that page is it better to have a 404 or actually see the content and a canonical?
The canonical handles the SEO issue. This is just people fetishizing how their URL looks.
Note: Goes without saying that's not a client friendly response.
2 -
No, definitely say they are fetishizing how their URL looks ahaha ?
1 -
The saga of MSE's SEO concerns continues!
- If someone goes to a nonsense URL, such as: https://staff.vanillaforums.com/asdfdjaksdjfkaldsl MSE agrees that it makes sense for the canonical to be https://staff.vanillaforums.com/home/file-not-found" (yay! ?)
- MSE's only remaining qualm with how we handle 404s is if there is a link to a page that is in fact deleted, i.e., if i tried to go to https://staff.vanillaforums.com/discussion/889/mses-seo-concerns after it's been (hypothetically) deleted, the canonical would still be https://staff.vanillaforums.com/home/file-not-found -- MSE's SEO team feels strongly that this is something that should be a self referencing canonical to signal to google that this has been deleted and is no good anymore.
I would imagine that differentiating between nonsense links and deleted links is not an easy task, but perhaps there's something to be done here?
0 -
Hey team,
MSE is still on my case for this.
>There's a canonical tag to https://mse.vanillacommunities.com/home/file-not-found on all 404 pages. 404 pages should canonicalise to themselves. I.e. this 404 page (https://mse.vanillacommunities.com/test) should have the canonical tag https://mse.vanillacommunities.com/test
>This issue relates to the 404 configuration on the areas of the site where it is possible to get a 404 - e.g. https://mse.vanillacommunities.com/test
>On the above example, this is clearly not a page that exists so it should return a 404 status, which it does. However, it's currently canonicalising to a singular 404 page (https://mse.vanillacommunities.com/home/file-not-found), which is wrong. If a page does not exist it should either have a self-referencing canonical or none at all.
Do we think this is something that we would consider doing as a prroduct improvement, or as paid custom work?
0 -
@Adrian @Adam Charron any comments on the above? getting lots of pressure from mse on this one
0 -
Those are fair points and they are right. I can’t comment if or when it would be fixed. They could use a redirect if they move a post - which could help.
0
