In light of the recent COVID-19 pandemic - OPTASY would like to offer DRUPAL website support for any Health Care, Government, Education and Non-Profit Organization(s) with critical crisis communication websites or organizations directly providing relief. Stay Safe and Stay Well.

How Do You Deal with Duplicate Content in Drupal? 4 Modules to Get this Issue Fixed

How Do You Deal with Duplicate Content in Drupal? 4 Modules to Get this Issue Fixed

by Adriana Cacoveanu on Jan 16 2019

Accidentally creating duplicate content in Drupal is like... a cold: 

Catching it is as easy as falling off a log.

All it takes is to:

  • further submit your valuable content on other websites, as well, and thus challenging Google with 2 or more identical pieces of content
  • move your website from HTTP to HTTPs, but skip some key steps in the process, so that the HTTP version of your Drupal is still there, “lurking in the dark”
  • have printer-friendly versions of your Drupal site and thus dare Google to face another duplicate content “dilemma”

So, what are the “lifebelts” or prevention tools that Drupal “arms” you with for handling this thorny issue?

Here are the 4 modules to use for boosting your site's immunity system against duplicate content.

And for getting it fixed, once the harm has already been made:

1. But How Does It Crawl into Your Website? Main Sources of Duplicate Content 

Let's get down to the nitty-gritty of how Drupal 8 duplicate content “infiltrates” into your website.

But first, here are the 2 major categories that these sources fall into:

  • malicious
  • non-malicious

The first ones include all those scenarios where spammers post content from your website without your consent.

The non-malicious duplicate content can come from:

  • discussion forums that create both standard and stripped-down pages (for mobile devices)
  • printer-only web page versions, as already mentioned
  • items displayed on multiple pages of the same e-commerce site

Also, duplicate content in Drupal can be either:

  • identical
  • or similar

And since it comes in “many stripes and colors”, here are the 7 most common types of duplicate content:

1.1. Scraped Content

Has someone copied content from your website and further published it? Do not expect Google to distinguish the copy from its source.

That said, it's your job and yours only to stay diligent and protect the content on your Drupal site from scrapers.

1.2. WWW and non-WWW Versions of Your Website

Are there 2 identical version of your Drupal website available? A www and a non-www one?

Now, that's enough to ring Google's “duplicate content in Drupal” alarm.

1.3. Widely Syndicated Content 

So, you've painstakingly put together a list of article submission sites to give your valuable content (blog post, video, article etc.) more exposure.

And now what? Should you just cancel promoting it?

Not at all! Widely syndicated content risks to get on Google's “Drupal 8 duplicate content” radar only if you set no guidelines for those third-party websites.

That is when these publishers don't place any canonical tags in your submitted content pointing out to its original source.

What happens when you overlook such a content syndication agreement? You leave it entirely to Google to track down the source. To scan through all those websites and blogs that your piece of content gets republished on.

And often times it fails to tell the original from its copy.

1.4. Printed-Friendly Versions

This is probably one of the sources of duplicate content in Drupal that seems most... harmless to you, right?

And yet, for search engines multiple printer-friendly versions of the same content translates as: duplicate pages.

1.5. HTTP and HTTPs Pages

Have you made the switch from HTTP to HTTPs?

Entirely?

Or are there:

  • backlinks from other websites still leading to the HTTP version of your website?
  • internal links on your current HTTPs website still carrying the old protocol?

Make sure you detect all these less obvious sources of identical URLs on your Drupal website.

1.6. Appreciably Similar Content 

Your site's vulnerable to this type of duplicate content “threat” particularly if it's an e-commerce one.

Just think of all those too common scenarios where you display highly similar product descriptions on several different pages on your eStore. 

1.7. User Session IDs 

Users themselves can non-deliberately generate duplicate content on your Drupal site. 
How? They might have different session IDs that generate new and new URLs.


2. 4 Modules at Hand to Identify and Fix Duplicate Content in Drupal

What are the tools that Drupal puts at your disposal to detect and eliminate all duplicate content?

2.1. Redirect Module

Imagine all the functionality of the former Global Redirect module (Drupal 7) “injected” into this Drupal 8 module!

In fact, you can still define your Global Redirect features by just:

  1. accessing the Redirect module's configuration page
  2. clicking on “URL redirects” 

How to Deal with Duplicate Content in Drupal: Global Redirect features

Image Source: WEBWASH.net

What this SEO-friendly module does is provide you with a user-friendly interface for managing your URL path redirects:

  • create new redirects
  • identify broken URL paths (you'll need to enable the “Redirect 4040” sub-module for that)
  • set up domain level redirects (use the “Redirect Domain” sub-module)
  • import redirects

Summing up: when it comes to handling duplicate content in Drupal, this module helps you redirect all your URLs to the new paths that you will have set up.

This way, you avoid the risk of having the very same content displayed on multiple URL paths.

2.2. Taxonomy Unique Module  

How about “fighting” duplicate content on your website at a vocabulary level?

In this respect, this Drupal 8 module:

  • prevents you from saving a taxonomy term that already exists in that vocabulary
  • is configurable for every vocabulary on your Drupal site
  • allows you to set custom error messages that would pop up whenever a duplicate taxonomy term is detected in the same vocabulary

2.3. PathAuto Module  

Just admit it now:

How much do you hate the /node125 type of URL path aliases?

They're anything but user-friendly.

And this is precisely the role that Pathauto's been invested with:

To automatically generate content friendly path aliases (e.g. /blog/my-node-title) for a whole variety of content.

Let's say that you want to modify the current “path scheme” on your website with no impact on the URLs (you don't want the change to affect user's bookmarks or to “intrigue” the search engines).

The Pathauto module will automatically redirect those URLs to the new paths using any HTTP redirect status.

2.4. Intelligent Content Tools      

Personalization is key when you strive to prevent duplicate content in Drupal, right? 

And this is precisely what this module here does: it helps you personalize content on your website.

How? Through its 3 main functionalities delivered to you as sub-modules:

  • auto tagging
  • text summarizing 
  • detecting plagiarized content 

Leveraging Natural Language Processing, this last sub-module scans content on your website and alerts you of any signs of duplicity detected.

Word of caution: keep in mind that the module is not yet covered by Drupal's security advisory policy!

3. To Sum Up

Setting a goal to ensure 100% unique content on your website is as realistic as... learning a new language in a week. 

Instead, you should consider setting up a solid strategy ”fueled” by (at least) these 4 modules “exposed” here. One that would help you avoid specific scenarios where entire pages or clusters of pages get duplicated.

Now, that's a far less utopian goal to set, don't you think?

Development

We do Web development

Go to our Web development page!

Visit page!

Recommended Stories

Top 10 Drupal Websites in Australia: From the Most Impressive to the Most Visited Ones
They're visually-appealing, highly popular, built on top of robust structures, so that they should "cope with" with huge amounts of traffic and content... These are but some of the things that the top 10 Drupal websites in Australia have in common. But which are they, more precisely? Australia's finest examples of websites using Drupal? We've done our research and put together a 10 best list for you. Let's dive right into it:   1. Australian Nuclear Science and Technology Organisation(ANSTO)   "Australia's research and development organization, and the center of Australian nuclear expertise..." (source: directory.gov.au) And the presentation could go on with other superlatives to "wow" you with:   Australia's most important science infrastructure (one of) Australia's largest public research organizations it sums up more than 1000 experts that use nuclear techniques to find the answer to various medical, environmental, and industrial challenges   A significant content load? A large team of content editors in need of the most robust tools to streamline their content management workflow with? The web project had Drupal 8 written all over it... And it quickly became one of the top 10 Drupal websites in Australia. Why Drupal? Because Drupal provides the best editing experience. Content managers are free to build new custom pages on their own, by just selecting and assembling pre-designed blocks. Not to mention that the "component-based" approach to design ensures consistency throughout the website and, implicitly... the best user experience, as well.    2. University of Technology, Sidney The University of Technology Sidney (UTS), Australia's #1 university and the 10th best one in the world, also has its website running on Drupal. Drupal is the platform that powers a lot of higher education websites worldwide, so... no surprise here, right? The reasons why UTS chose Drupal are obvious:   their multi-site structure incorporates no less than 150 UTS sub-sites the "cluster" of mini-sites needed a central content hub and a unified platform the internal content team needed enough freedom and flexibility to create, edit, and publish content across the entire network of sub-sites they needed to integrate lots of third-party systems (e.g. systems sourcing course information)    In short:   Drupal makes the perfect choice for multi-site setups it integrates seamlessly with third-party apps it provides the best editorial experience   3. The Australian Government       Undoubtedly one of the top 10 Drupal websites in Australia.  "The Government of Australia is standardizing on Drupal for hundreds of government departments and agencies across Australia, starting with the country’s homepage. Drupal enhances digital services to millions of citizens." (source: Drupal.org)   Why Drupal? For multiple reasons:   Drupal's ideal for multi-site architectures Drupal provides a granular permission system it integrates with lots of third-party services and solutions (payment systems, citizen service centers, various "latest news" sources)  it's one of the most secure CMS platforms: Drupal has its own dedicated Security Team and a foolproof process for monitoring, identifying, and publishing any security issues it's flexible, giving unmatched freedom to content editors and website admins: anyone from the internal team can update content on the website quick and easy Source: Drupal.org   4. Bunjil Place     The "home of arts and entertainment in the City of Casey" is Drupal-powered.  And it's probably one of the most visually impressive examples of Drupal websites from Australia. A high-trafficked website, that "bids on" Drupal 8 to provide an interactive user experience. Source: PreviousNext.com Now, if we were to list just some of the strongest reasons why Drupal was the best option for this Australian website:   the diversified content hosted on the website (text, video, eye-catching images), that falls into multiple sections and subsections the need for a new responsive design the need to incorporate some powerful user-oriented functionalities (e.g. visitors can easily book spaces in the palace for their future events)   Drupal was the only choice here... 5. The Australian Drupal's already famous for being the best CMS solution for news websites. And the CMS platform is behind the online presence of "The Australian" — Australia's national news brand — as well. A high-trafficked, content-packed website, with content that needs to be updated on the fly, on a regular basis.  They couldn't have opted for another platform than Drupal... 6. Australian Conservation Foundation, One of the Top 10 Drupal Websites in Australia   Australia's national environment organization — over 600,000 people — has its website running on Drupal. It's a campaign-driven site, one of the most popular Drupal-based websites in Australia, that leverages the CMS platform's top features to:   manage all their content workflows with great ease engage their member database and anyone willing to address urgent environmental problems via social networking, donation processing, event management, analytics, fundraising   A website that needed to withstand high amounts of traffic and to support complex content workflows: Drupal was the most suitable CMS solution.   7. Australian War Memorial "The Memorial's purpose is to commemorate the sacrifice of those Australians who have died in war or on operational service and those who have served our nation in times of conflict." (source: awm.gov.au) One of the top 10 Drupal websites in Australia that turned this CMS platform into a reliable publishing tool for all the historians, researchers, and designers accessing it. We're talking here about a content-heavy website, hosting content that ranges from:   war-related material to extensive archive to a database of unique collections of film footage, artworks, sound recordings to articles on Australia's military history   And where there's rich and varied content, that needs to be handled with ease by various internal teams, there's Drupal, as well. Drupal with its:   robust inline content creation and editing tools sophisticated access control system   8. Charles Darwin University  One of Australia's public universities and one of the world's most reputed universities (among the top 2% universities on the globe) has its website running on Drupal. And for all the self-evident reasons:   they needed to provide internal users with the best experience: Drupal enables content editors to create their custom course pages by simply putting together pre-built components they also needed to provide an intuitive search experience to all the external users: Drupal makes it easy for students to find, review courses, and sign up   9. International Business Times Australia   The Australian version of this digital news publication is built on Drupal. But is it of any surprise that the go-to source of financial news in Australia has chosen this CMS platform to build its online presence on? After all:   news publications are Drupal's "specialty" Drupal's already famed for powering some of the most content-intensive websites in the world  Drupal's gained a reputation for being the CMS that provides the best content editing experience: the internal team at International Business Times Australia can create, update, and publish new content on the go   10. Car News Australia       One of the top 10 Drupal websites in Australia, that stands out from the crowd as a high-trafficked one. Its complex content ecosystem has turned Drupal into the Carsguide team's only option. The site's an intricate network of 7 different sections — "buy + sell", "reviews", "news", "advice"... — that unfold, each, into "clusters" of multiple sub-sections. A content network that called for a CMS with particularly powerful content creation and editing features. A content-rich architecture that's now powered by Drupal.   The END! We're curious now: what other famous brands are on your own top 10 Drupal websites in Australia? Photo by Iván Lojko on Unsplash  ... Read more
Adriana Cacoveanu / Apr 03'2020
Drupal Layout Builder vs Gutenberg: How Do You Know Which Page Builder’s Right for You? 
Drupal Layout Builder vs Gutenberg: which of these 2 next-generation site-building tools to use to create your page layouts?  Or, better said, to empower your content editors to design great-looking pages on their own.  In order to help you make an informed decision, I've made an honest “Layout Builder modules vs Gutenberg editor module” comparison.  Therefore, in this post you'll find your answers to the following questions:    What issues with the editing experience in Drupal do these 2 new page layout builders come to address? What is the Gutenberg Editor? What are Drupal Gutenberg's main strengths? What are its limitations? What is the Layout Builder for Drupal? What are the Drupal Layout Builder's main strengths? What are its weaknesses? When would you use one Drupal page builder over the other?    1. Drupal Layout Builder vs Gutenberg: What Limitations Do They Address?  Or, to put it this way: what makes a good content editing experience?  Here are some of the editors' main requirements:    to be easy to use (with an intuitive drag and drop interface) to enable them to create custom page layouts on the fly, with zero CSS or HTML knowledge to be ideally flexible, enabling them to adjust the pages to their needs    Now, if we put editors' requirements against the "reality" of the editing experience in Drupal, here are the limitations we can identify:    it doesn't provide a decent level of visual design it compromises branding for flexibility or the other way around it requires some HTML experience it doesn't go beyond the level of a basic WYSIWYG editor it doesn't allow them to design fully customed page layouts it doesn't provide a 100% intuitive user interface it still makes editors dependant on the site builders/Drupal developers in their teams And these are precisely the problems that these 2 Drupal page builders "promise" to solve.    2. Introducing: The Gutenberg Drupal 8 Module   "Ever since the “Gutenberg” editor was released for WordPress and gained popularity we’ve seen a surge of “Easy to use” drag and drop interfaces for various CMSes."  (source: codekoalas.com) The Gutenberg Editor module is no more than WordPress's layout system integrated with Drupal.  What sets it apart from the other tools for creating page layouts in Drupal?  Its block-based approach.  Creating content in Drupal becomes a matter of selecting and assembling multiple blocks.  Each block stands for an individual piece of content: a separate block for a button, another separate block for an image, another one for a column, for a heading...  Furthermore, you get to create content inline. To edit the media content, text, and blocks on the page that you're building.  In short, you can take the Drupal Gutenberg module as a convenient "block editor".  You (or your editors) can put together and visualize your page layout in one large form field. That, instead of editing every single form field that makes up a page.    3. 4 Reasons to Choose Drupal Gutenberg  Let's say that you're looking for a site building tool to build a... landing page (or one that would empower your content editors to create it).  Why would you go with the Gutenberg content editor for Drupal 8?  Here are 4 good reasons to choose it:    it provides you with lots of predesigned elements out of the box you can see what you're building on the editor side it doesn't require you to be experienced in working with CSS or HTML you can design your own blocks or use blocks created by other open-source developers    4. 6 Reasons Why You Might Hesitate to Choose Gutenberg  Any honest Drupal Layout Builder vs Gutenberg comparison must include the 2 tools' main weaknesses, as well, right?  So, here's why you may want to think twice before choosing the Gutenberg content editor in Drupal 8:    the way it stores data could be... better it doesn't excel when it comes to layout flexibility: you can align your blocks left, right, center, but you can't add custom margins, for instance the style options that it provides are quite limited, one of the main drawbacks highlighted in any Gutenberg vs page builder comparison updating a page created with Gutenberg is particularly overwhelming the visual editing experience is not one of the best ones: visualizing your layout is a bit more difficult you cannot reuse your Gutenberg blocks and the page layouts that you create with this editor    Tip: when it comes to the poor visual experience that you get with the Gutenberg Editor, you can always improve it with page builder plugins. There are some pretty good ones out there that guarantee you a 100% visual design.   5. Introducing: The Drupal Layout Builder Module (Core) "Drupal 8's Layout Builder allows content editors and site builders to easily and quickly create visual layouts for displaying content. Users can customize how content is arranged on a single page, or across types of content, or even create custom landing pages with an easy to use drag-and-drop interface." (source: Drupal.org) The Layout Builder for Drupal enables you to create layouts by putting together separate form fields and block entities. No coding know-how needed.  No need to tweak the templates in the theme's source code.  In short, it "spoils" you with a drag-and-drop type of visual editing experience. You can put together structured content quick and easy.  Source: Drupal.org "But how exactly does it work?"  Here's how you create a custom layout with this Drupal page builder in 5 quick steps:  enable the module go over to the "Manage Display" tab of your target content type click the "Use Layout Builder" checkbox click the "Manage Layout" button then, you're free to use your Layouts and Blocks (after you've created them), dropping them in the defined regions of the page that you're building    6. 7 Reasons to Choose the Layout Builder in Drupal 8  Here's a legitimate question that this Gutenberg Editor vs Layout Builder in Drupal 8 comparison should respond to:  “Why would I want to use Drupal Layout Builder?”  The best answers would be:    you can create your content blocks and just drop them on the regions of your page layout you get to (re)arrange otherwise static fields the way you want to you're free to add views, blocks, widgets, to any section of your page the content that you create is specific to that page type, so all the pages of that type will have the same layout you get to override the default layout to create fully customed pages your layout is rendered on the web page in real-time you're free to incorporate blocks in the main content region 7. 5 Reasons Why You Might Hesitate to Choose the Layout Builder  When comparing Drupal Layout Builder vs Gutenberg, you inevitably get to the point where you ask yourself:  "What are the Layout Builder's main weaknesses?"  Now, here's why you wouldn't want to jump on it right away:    you can use it only on the main content area of your page unlike the Gutenberg Drupal 8 module, the Layout Builder doesn't ship with prebuilt elements for you to just choose from and assemble with Drupal Layout Builder, you need to create your blocks and page type first; you can't "get away with" choosing user-contributed blocks it's not always easy to see what precisely gets overridden you can't rely on a granular permissions system for overriding individual pages    8. When to Use Drupal Gutenberg: 4 Typical Use Cases  You'd want to use the Gutenberg Drupal 8 module if:    your Drupal website's a content-packed one you need to build a landing page you need to create a simple layout: Gutenberg doesn't really shine when it comes to style options you need to create content that fits perfectly into your theme's looks: Drupal Gutenberg pulls right from your theme's styles    To sum up, designing landing pages are Gutenberg's most typical use case.  Use it to add just some extra design (nothing too "extravagant") to an "About Me" page, for instance.   9. When to Use the Drupal Layout Builder: 2 Typical Use Cases  You'd want to use the Layout Builder for Drupal if:    it's a field-intensive page that you're building: a product page, a case study, etc. you have synced content on that page (from Salesforce, for instance)   The END!  Your turn now: have you tried any of these Drupal page builders?  Which one best serves your content goals?  What other pros and cons would you have added to this Drupal Layout Builder vs Gutenberg comparison?  Let us know in the comments below. Photo by Jason Dent on Unsplash  ... Read more
Silviu Serdaru / Mar 25'2020
Drupal Voice Search: How to Set Up Search Based on Voice Recognition on Your Drupal Site
Drupal voice search has gone from trend to standard these days.  So, you can't help ask yourself:  "How do I enable search based on voice recognition on my Drupal website?"  And that's precisely the question that I'm about to answer in this blog post.  That, along with all the other ones that must be "haunting" you:    How does voice search impact my revenues?  How do I optimize my website content for voice search queries?  How do I set up a custom voice search engine on my Drupal site?  Can I even build a voice-powered app with Drupal? How?    1. How Does Voice Search Impact Your Traffic, Conversions, Revenues?  “Voice Shopping Set to Jump to $40 Billion By 2022, Rising From $2 Billion Today“ (source: prnewswire.com) How does making your Drupal website voice search-friendly translate into higher conversion rates?  The logic is simple:    it's more convenient for users to speak to the search bar than to type in their queries in order to find the needed products or services by allowing them this convenience, you improve their experience on your website and that'll bubble up to your web traffic, conversions... revenues  "Becoming voice-first is going to be the new mobile first" (DrupalCon Nashville 2018)  In other words:    understanding how search engines provide accurate answers to users' vocal queries optimizing your content to the patterns that you will have identified    … will have a huge impact on the user experience on your Drupal site.  Source: Gartner.com And great user experience is what sets high ranking websites apart from... the rest.    2. Here's How You Make Your Content Cater to Voice Search Queries  Spoiler alert: I'll be pinpointing just the main techniques to apply for optimizing your content for voice search.  If you feel like delving deeper into this topic, we have a full post focused solely on actionable tips for optimizing your website for voice search. That being said, here's the list of essential voice search SEO techniques to implement:   2.1. Incorporate Short, Simple, Clear Answers Into Your Content 30 words at most, according to Backlinko's research.   2.2. Fine-Tune the FAQ Page on Your Drupal Site This page alone plays a key role in your voice search optimization strategy. It's there that search engines often “extract” their answers from.   2.3. Use Natural Language When Writing Your Content A user would speak to a search bar as he/she would ask a question to a person. Which is significantly different from him/her interacting with a text search box via a robotic keyword search entry.   2.4. Configuring Drupal for Voice Search: Optimize Your Content for Google's Featured Snippets I'm not going to get into details here, but there are lots of highly valuable MOZ blog posts on this topic that I recommend you have a look at.   2.5. Center Your Drupal Voice Search Strategy Around Question Keywords “How to...”, “What to...”, “Why use...”   2.6. Make Sure to Include (Really) Long-Tail Keywords Into Your Content  5+ word keywords, I mean.   3. Set Up a Drupal Voice Search Engine in Just a Few Steps  How do you create a search system based on voice on your Drupal website?  Image by Kaufdex from Pixabay Here are the easy steps to creating a custom voice search engine with Expertrec:    First of all, you need a google tag manager account, a Gmail id, access to your Drupal admin dashboard and a valid sitemap    Set up an account on https://cse.expertrec.com??platform=cse using your Gmail ID   Enter your website's URL login to your Drupal admin panel and install google tag manager: https://www.drupal.org/project/google_tag   Enter your google tag manager ID and save the configuration   Access https://cse.expertrec.com/csedashboard/home/code and copy that code   Add the first part of the code in your control panel, using Google tag manager   Use the custom HTML option and configure the trigger to place it on all the web pages on your Drupal site where you want the voice search enabled   Go to Structure-> Block layout-> Header ->Place block-> Add custom block-> click on source and enter: <ci-search></ci-search> (opt for "full HTML"). It's this code that will display a search box in your website's header   Hit the Save button and finally, navigate to UI customization->features-> enable voice search    And that's it: 9 simple steps to add voice search to your Drupal site.    4. Building Your Own Voice App: Why Would You Want to Use Drupal?  Now, let's say that you've optimized your Drupal website for voice search, improved your search ranking and given your traffic a major boost.  And you've decided to go further and... build your first voice app, as well.  Why would you use Drupal to build it?  Here are some of the most solid reasons:    Drupal provides you with a sophisticated user permissions and access control system it allows you to restrict access to content it provides you with fieldable content it ships with a pre-built front-end Drupal it enables you to put together complex content workflows it provides multi-language support out of the box it provides you with an API module    Note: in the end, it's your use case, along with all the specific feature needs deriving from there, that should influence your final decision.  But how would your Drupal voice app work, more precisely?  Here's the whole "mechanism" in the back-end, summed up to 4 stages:    the user talks to the voice assistant (Alexa, Siri, Cortana...) the voice assistant talks to Drupal Drupal provides an answer to the voice assistant the voice assistant delivers the answer to the user    Chatbot API is the module that makes communication between Drupal and the AI voice assistant possible. It integrates, out of the box, with Dialogue Flow and Alexa. I won't go into detail on the basics of a voice-powered app and the fundamentals of a VUI design. Yet, I recommend you watched the DrupalCon session on Powering an Alexa App with Drupal. It includes a handful of useful tips and actionable information on building a Drupal-powered voice search. In this chapter, my sole intention was to outline those Drupal features and functionalities that make it the best candidate for a voice app.    The END!  I won't hide it: I'm curious whether you've already got your Drupal website voice-search friendly or not just yet.  What strategies have you implemented so far to make your content more findable for vocal queries?  Have you set up your own Drupal voice search engine? Are there any obstacles that Drupal challenged you with? That made the whole optimization for voice search more complex than expected?  Let us know in the comments below.  Photo by Matt Botsford on Unsplash  ... Read more
Adriana Cacoveanu / Mar 23'2020