Drupal Planet

October's Most Interesting New Drupal Modules and Themes

Wayne Eaker — Fri, 06 Nov 2015 18:05:44 +0000

October's Most Interesting New Drupal Modules and Themes

Some of the most interesting new modules and themes I saw posted to drupal.org in October:

Views Manager [sandbox]

Gives you vertical tabs on the Views admin page, separating Views by their tags.

BigPipe

Allows for speedier page loads for authenticated users. First you send the fast, cached parts of the page, then fill in the expensive parts when they are ready.

Accelerated Mobile Pages (AMP)

Make your Drupal site ready for Accelerated Mobile Pages. Right now, it only adds the AMP javascript library, but the plan is to serve AMP pages directly from Drupal.

This module creates multiple paths for the same node using Organic Groups and Pathauto. It’s not best practice for SEO, but it’s nice in situations where content is posted to multiple groups, and you want the node path to reflect the group context the user is currently in.

Menu Link Field

Provides a field similar to the menu link form on node forms but for any entity type.

Human Behavior

An anti-bot technology that looks for mouse movement and mobile device touch events to determine if a bot is filling out a form.

Views Numeric Range Filter [sandbox]

It gives you a numeric filter for Views that includes a +/- tolerance. So you can say 100 +/- 10%, and it will give you results between 90 and 110. Neat!

Address Field Lookup [sandbox]

Integrates the Address Field module with validation services to correct address formatting issues.

Features Export Explode

Breaks your features into a bunch of different files, which makes it easier to see what changed when you review a git commit. For example, it breaks each view out into its own file.

Pistachio

An example theme for Drupal 8 that shows you how to use the new features of the D8 theme system. Heavily commented to explain your options.

Commerce Cart Link

Provides a new field formatter for Add to Cart fields attached to Commerce products and product displays that is a simple link, rather than a form. This way users can buy a product without invoking the full Form API on each product item.

CRUD log

Captures and logs all node CRUD operations, “create”, “read”, “update” and “delete”. This module logs the events in a custom table and exposes the data to Views. It allows you to select which operations will be captured, so you can turn off read events if that is too verbose.

User Moderation [sandbox]

A user moderation system that uses votes from other users to decide if a user is a spammer or not.

Responsive Menu Combined

Allows you to combine multiple menus into the responsive hamburger menu with tabs in it.

Mailsuite [sandbox]

Provides a unified administration interface for managing the templates of all the emails you send from your Drupal site.

Twig PHP Filter

Ha! If you love doing everything wrong and putting PHP code in your template files, this is the Drupal 8 module for you. If you’re slightly less crazy and only want to query the DB from your twig files, then the new Little Bobby Tables module may be a “better” choice.

Wayne EakerNovember 6, 2015

Drupal Planet

September's Most Interesting New Drupal Modules

Wayne Eaker — Thu, 01 Oct 2015 18:00:04 +0000

September's Most Interesting New Drupal Modules

Some of the most interesting new modules I saw posted to drupal.org in September:

Views Advanced Routing

(for Drupal 8) Allows you to specify the routing configuration YAML for a Views page. Meaning, you can use custom access control callbacks, default parameters, etc. Sweet!

Commerce Responsive UI

Provides replacement interfaces for the parts of Drupal Commerce that are table dependent and non-mobile responsive by default. These include Responsive Cart, Responsive Checkout, and Responsive User Facing Orders.

Drupal 8 Contrib Porting Tracker

Not a module, but a centralized place for tracking the Drupal 8 porting status of contributed projects (modules, themes, distributions). The best place to find out that the Bad Judgement module is ready for D8!

Advanced Image Crop

This image field cropper lets the user do a different crop in each of the image styles configured by the admin. You better have some saavy users to comprehend this, but if you do, it looks awesome.

Webform Replay [sandbox]

Extends the Webform module by adding an option to “replay” selected webform values in situations where multiple webform submissions per user are allowed, and some of that information is likely to be repeated on each submission. By enabling webform replay for these fields, the user only needs to complete them for the initial webform submission, and on subsequent entries these fields will be pre-populated with the values from the previous submission.

Forbidden File Format

Flips the file field extension checking around so that you can allow all types of files except the extensions specified. So you could deny .js, .exe, .bat, and .com, but allow other types.

Tableau WDC [sandbox]

Tableau 9.1 includes a new Web Data Connector feature, which lets you build connections to data accessible over HTTP with JSON data and REST APIs. This module attempts to bridge the gap between Drupal and Tableau by adding a new views plugin (tableau_wdc) which renders content as a JSON with some extra meta information needed by Tableau. Once you have created your endpoints, you can add the tableau-wdc block to any page and it will automatically render a button for each data source together with all the necessary scripts to parse and prepare the data for import.

Nuke Drupal Frontend

Allows you to completely disable frontend HTML access to a Drupal site, for when you’re building a headless site, and you’re not using the Drupal-provided frontend.

Doubtfire [sandbox]

An alternative to the Masquerade module, with some useful UI additions.

Gmail Connector [sandbox]

Lets users view their Gmail inbox and messages in Drupal using the Gmail RESTful API.

Wayne EakerOctober 1, 2015

Drupal Planet

DrupalCamp Michigan 2015: January 31 at Henry Ford College

Wayne Eaker — Wed, 17 Dec 2014 17:49:05 +0000

DrupalCamp Michigan 2015: January 31 at Henry Ford College

Attention Michigan Drupalers! Mark your calendars for DrupalCamp Michigan, Saturday January 31 at Henry Ford College in Dearborn. For only $15, you get a full day of Drupal presentations, discussions and lunch!

Online registration/tickets will be available soon, and we’re currently accepting session proposals. If you’ve got an interesting presentation to share, please post it to the website here: https://2015camp.michigandrupal.com. Session selections will be made in early January.

Interested is sponsoring the camp and getting your name out to the local Drupal community? Sponsorship information is available here: https://2015camp.michigandrupal.com/sponsors.

Wayne EakerDecember 17, 2014

Drupal Planet DrupalCamp Michigan

Decoupling Your Backend Code from Drupal (and Improving Your Life) with Wrappers Delight

Wayne Eaker — Wed, 03 Dec 2014 19:48:07 +0000

Decoupling Your Backend Code from Drupal (and Improving Your Life) with Wrappers Delight

If you've ever written a lot of custom code for a Drupal site, then you know it can be a tedious and error-prone experience. Your IDE doesn't know how Drupal's data structures are organized, and it doesn't have a way to extract information about configured fields to do any autocomplete or check data types. This leads to some frustrations:

You spend a lot of time typing out by hand all the keys in every array of doom you come across. It's tedious, verbose, and tiring.
Your code can contains errors your IDE won't alert you to. Simple typos can go unnoticed since the IDE has no idea how the objects and arrays are structured.
Your code is tightly coupled to specific field names, configured in the database. You must remember these, because your IDE can't autocomplete them.
Your code is tightly coupled to specific field types. (If you start off with a text field and then decide to switch to an email field, for example, you will find the value is now stored in a different key of the data array. You need to update all your custom code related to that field.)
It can be easy to create cross-site-scripting vulnerabilities in your code. You need to keep in mind all the field data that needs to be sanitized for output. It only takes one forgotten spot to open your site to attacks.

Wrappers Delight (https://www.drupal.org/project/wrappers_delight) is a development tool I've created to help address these issues, and make my life easier. Here's what it does:

Provides wrapper classes for common entity types, with getters and setters for the entities' base properties. (These classes are wrappers/decorators around EntityMetadataWrapper.)
Adds a Drush command that generates wrapper classes for the specific entity bundles on your site, taking care of the boilerplate getter and setter code for all the fields you have configured on the bundles.
Returns sanitized values by default for the generated getters for text fields. (raw values can be returned with an optional parameter)
Allows the wrapper classes to be customized, so that you can decouple your custom code from specific Drupal field implementation.

With Wrappers Delight, your custom code can be written to interface with wrapper classes you control instead of with Drupal objects directly. So, in the example of changing a text type field to an email type field, only the corresponding wrapper class needs to be updated. All your other code could work as it was written.

But wait, there's more!

Wrappers Delight also provides bundle-specific wrapper classes for EntityFieldQuery, which allow you to build queries (with field-level autocomplete) in your IDE, again decoupled from specific internal Drupal field names and formats. Whatever your decoupled CRUD needs may be, Wrappers Delight has you covered!

Getting Started with Wrappers Delight

To generate wrapper classes for all the content types on your site:

Install and enable the Wrapper Delight module.
Install Drush, if you don't already have it.
At the command line, in your Drupal directory, run drush wrap node.
This will generate a new module called "wrappers_custom" that contains wrapper classes for all your content types.
Enable the wrappers_custom module, and you can start writing code with these wrapper classes.
This process works for other entity types, as well: Users, Commerce Products, OG Memberships, Messages, etc. Just follow the Drush command pattern: drush wrap ENTITY_TYPE. For contributed entity types, you may need to enable a submodule like Wrappers Delight: Commerce to get all the base entity properties.

Using the Wrapper Classes

The wrapper classes generated by Wrappers Delight have getters and setters for the fields you define on each bundle, and they inherit getters and settings for the entity's base properties. The class names follow the pattern BundlenameEntitytypeWrapper. So, to use the wrapper class for the standard article node type, you would do something like this:

$article = new ArticleNodeWrapper($node);
$body_value = $article->getBody();
$image = $article->getImage();

Wrapper classes also support passing an ID to the constructor instead of an entity object:

$article = new ArticleNodeWrapper($nid);

In addition to getters that return standard data arrays, Wrappers Delight creates custom utility getters for certain field types. For example, for image fields, these will all work out of the box:

$article = new ArticleNodeWrapper($node);
$image_array = $article->getImage();
$image_url = $article->getImageUrl();
$image_style_url = $article->getImageUrl('medium');
$absolute_url = $article->getImageUrl('medium', TRUE);

// Get a full  tag (it's calling theme_image_style
// under the hood)
$image_html = $article->getImageHtml('medium');

Creating New Entities and Using the Setter Methods

If you want to create a new entity, wrapper classes include a static create() method, which can be used like this:

$values = array(
  'title' => 'My Article',
  'status' => 1,
  'promote' => 1,
);
$article = ArticleNodeWrapper::create($values);
$article->save();

You can also chain the setters together like this:

$article = ArticleNodeWrapper::create();
$article->setTitle('My Article')
  ->setPublished(TRUE)
  ->setPromoted(TRUE)
  ->save();

Customizing Wrapper Classes

Once you generate a wrapper class for an entity bundle, you are encouraged to customize it to your specific needs. Add your own methods, edit the getters and setters to have more parameters or different return types. The Drush command can be run multiple times as new fields are added to your bundles, and your customizations to the existing methods will not be overwritten. Take note that Wrappers Delight never deletes any methods, so if you delete a field, you should clean up the corresponding methods (or rewrite them to get the data from other fields) manually.

Drush Command Options

The Drush command supports the following options:

--bundles: specify the bundles to export (defaults to all bundles for a given entity type)
--module: specify the module name to create (defaults to wrappers_custom)
--destination: specify the destination directory of the module (defaults to sites/all/modules/contrib or sites/all/modules)

Packaging Wrapper Classes with Feature Modules or Other Bundle-Supplying Modules

With the options listed above, you can export individual wrapper classes to existing modules by running a command like the following:

drush wrap node --bundles=blog --module=blog_feature

That will put the one single wrapper class for blog in the blog_feature module. Wrappers Delight will be smart enough to find this class automatically on subsequent runs if you have enabled the blog_feature module. This means that once you do some individual exports, you could later run something like this:

drush wrap node

and existing classes will be updated in place and any new classes would end up in the wrappers_custom module.

Did You Say Something About Queries?

Yes! Wrappers Delight includes a submodule called Wrapper Delight Query that provides bundle-specific wrapper classes around EntityFieldQuery. Once you generate the query wrapper classes (by running drush wrap ENTITY_TYPE), you can use the find() method of the new classes to execute queries:

$results = ArticleNodeWrapperQuery::find()
  ->byAuthor($uid)
  ->bySomeCustomField($value1)
  ->byAnotherCustomField($value2)
  ->orderByCreatedTime('DESC')
  ->range(0, 10)
  ->execute();

The results array will contain objects of the corresponding wrapper type, which in this example is ArticleNodeWrapper. That means you can immediately access all the field methods, with autocomplete, in your IDE:

foreach ($results as $article) {
  $output .= $article->getTitle();
  $output .= $article->getImageHtml('medium');
}

You can also run queries across all bundles of a given entity type by using the base wrapper query class:

$results = WdNodeWrapperQuery::find()
  ->byAuthor($uid)
  ->byTitle('%Awesome%', 'LIKE')
  ->execute();

Note that results from a query like this will be of type WdNodeWrapper, so you'll need to check the actual bundle type and re-wrap the object with the corresponding bundle wrapper in order to use the bundle-level field getters and setters.

Wrapping Up

So, that's Wrappers Delight. I hope you'll give it a try and see if it makes your Drupal coding experience more pleasant. Personally, I've used on four new projects since creating it this summer, and it's been amazing. I'm kicking myself for not doing this earlier. My code is easier to read, WAY easier to type, and more adaptable to changes in the underlying architecture of the project.

If you want to help me expand the project, here are some things I could use help with:

Additional base entity classes for common core and contrib entities like comments, taxonomy terms, and Workflow states.
Additional custom getter/setter templates for certain field types where utility functions would be useful, such as Date fields.
Feedback from different use cases. Try it out and let me know what could make it work better for your projects.

Post in the issue queue (https://www.drupal.org/project/issues/wrappers_delight) if you have questions or want to lend a hand.

Wayne EakerDecember 3, 2014

Drupal Drupal Planet Module Development Wrappers Delight

DrupalCamp Michigan 2013: October 12 in Ann Arbor

Wayne Eaker — Wed, 02 Oct 2013 14:12:30 +0000

DrupalCamp Michigan 2013: October 12 in Ann Arbor

Michigan Drupalers! Let’s get together for DrupalCamp Michigan, Saturday October 12 in Ann Arbor. A great program has just been announced, including sessions on security, integrating Drupal with external systems, mobile app development, and Drupal 8. Registration is $15, which includes lunch and t-shirt. It’s a small venue, and there are limited tickets, so register soon to make sure you can attend.

Drupalers from neighboring states, you’re invited too! Yes, even Buckeyes. (we’ll try to be nice)

The full program is up at: https://2013.drupalcampmi.org/program
Tickets are available here: https://2013.drupalcampmi.org/register

The Scheduled Sessions

Keynote: The Power of Drupal in Higher Education
Chris Hartigan, General Manager for Higher Education at Acquia

Social in Your Site, The Right Way
Kevin Champion

Secure Your Site
Matt Farina

Editors: SWIM (show what I mean) versus WYSIWYG
Kieran Mathieson

DrupalGap - A Mobile Application Development Kit for Drupal
Tyler Frankenstein

Building a Curriculum Management and Publication System using ERP Data and Web Services
Micah Webner

Drupal and the LMS with LTI
Matthew Radcliffe

Drupal 8: What You Need to Know
Wayne Eaker

We will also have an after-party for networking, but we’re still working out the logistics of that.

Wayne EakerOctober 2, 2013

Drupal Planet DrupalCamp Michigan

Creating an iPhone App for a Drupal Website

Wayne Eaker — Tue, 10 May 2011 12:00:00 +0000

Creating an iPhone App for a Drupal Website

Recently, Zengenuity released the LA2M to Go iPhone app, a companion to the Lunch Ann Arbor Marketing website. The LA2M website is one we built on Drupal 6, and it uses custom content types, Views and Imagecache for displaying upcoming and archived event information for the group. In creating our app, I wanted to build a system to synchronize the mobile device with the website, since event information changes on at least a weekly basis. It took a lot of research and testing various techniques (many of which of which didn’t end up working), but finally I worked out how to have the app to pull all its information automatically from the Drupal site. I hope that by publishing this blog post, I can speed up this process for you, and save you some frustration. There isn’t room here to go into every detail, but I’ll discuss the major frameworks I used, and I’ll point out some important issues that wasted a lot of my time.

The Titanium Mobile Framework

LA2M to Go is built with the Titanium Mobile Framework by Appcelerator. The Titanium SDK is free, and it allows you to write native iPhone apps in web developer friendly Javascript, instead of Objective-C. It also allows you to compile the same source code as a native Android app, though this is something I haven’t tried yet. To get started with Titanium, download the SDK here. This will provide you with the Titanium Developer application you need to compile your app. You will also need to have developer tools for the target mobile platform you plan to work with. So, if you are writing an iPhone app, you need to have an Apple iOS developer account, and download XCode and the iOS SDK from Apple. Android developers will need to get the Android SDK from Google.

When coding for Titanium, you have to use your own text editor or IDE, and then load your project into the Titanium Developer application. Titanium-based apps are compiled in a two-step process. First, your code is rewritten by Titanium Developer as native iPhone or Android source code. Then, command line tools provided by the iPhone and Android SDKs are used to compile the native code and install it into the simulator or directly onto your mobile device.

One word of warning: This process absolutely sucks. It’s the worst compilation and debugging system I’ve ever experienced. When working with Titanium, be prepared to write a lot of output log statements, because that’s how you’ll be doing much of your debugging. Also, the Titanium platform is seriously lacking in documentation, and the docs that do exist are difficult to use and search. There are very few example code snippets in the API docs, which means you need to search the developer forums. In the forums, code is often wrong or out-of-date, as there’s no way to know which version of the Titanium API it’s designed for. (The API has changed significantly from its initial release.) Appcelerator recently purchased Aptana, the makers of the excellent Aptana Studio IDE. So, I’m praying they ditch the Titanium Developer application for a more fully-featured IDE, with debugging tools and built-in documentation. This would help developers immensely. (Update (5/17/11): They have now done exactly this: https://developer.appcelerator.com/blog/2011/05/major-titanium-updates.html)

That said, I’ve written code in Objective-C with XCode before, and while XCode has both excellent debugging features and documentation, the speed at which you I create robust apps with Titanium, in a language that is more familiar to me as a web developer, means I’ll coming to back to it for my next app, despite its failings.

The DrupalCon Chicago App

If you were at DrupalCon Chicago, you probably used the conference app that was available for iPhone and Android. That app was created in Titanium, by the folks at Palantir. Fortunately for all of us developing Drupal-connected apps, they have made the app’s source code available, and their blog post about it is very informative.

There isn’t a lot of technical documentation about how expand or change the app, but after spending some time with it, I was able to figure out the key details:

Schema Definitions

The app is designed to import Drupal nodes and store them in an SQLite database on the device. This way the app can be used offline. In the code, there is a schema definition designed for the DrupalCon event node type in the Resources/drupalcon/entity.js file. (You will also find a schema for Drupal.org users in there, too.) If you are building a new app from this source code, you need to modify their schema to match whatever fields your content types have. They have chosen to call their event schema “node”, but you can rename this to whatever you want, and add additional schemas for more types. If you are adding new schema types (or changing the name of them), you need to make entries for them in the Resources/drupal/entity.js file. There’s a section at the top that starts with

main: {
...
types {
[a list of types and primary keys is here]

Add your new schema names to this list, following the pattern of the others in that section.

One thing I found useful when modifying the schema were the two commented out lines in app.js that clear the DB on every app launch.

//Drupal.entity.db('main', 'node').initializeSchema();
//Drupal.entity.db('main', 'user').initializeSchema();

These obviously need to be commented out in production, but are useful when changing the schema. (Adjust the parameters to fit your schema names.) Another helpful tool you will want to have is some sort of SQLite database browser. I ended up using the SQLite Manager Firefox extension, though, I didn’t especially enjoy it. It works, but it’s kind of clunky. If you are building and iPhone app and working in the simulator, the SQLite database is difficult to find. Here’s where it’s at:

/Users/[your_username]/Library/Application Support/iPhone Simulator/[sdk_version]/Applications/[application_guid]/Library/Application Support/database/main.sql

Yeah...it’s really buried down there. One of those folders is hidden too, so you will not find that database file with Finder. You just have to know where it is.

Renderers

If you look in Resources/drupalcon/drupalcon.js, you will see a series of sections that start like this:

DrupalCon.renderers.session = function(session) {

These sections define “renderers” for different entities that you have in the SQLite database. They don’t do anything on their own. You have to call it yourself after you query the DB. But, when you look around in the app code, you’ll see that having a function that renders an SQLite database row as a TableViewRow is very handy. It’s sort of like defining a theme template for a particular node type. If you have added fields and types to the database schema, you will need to update the renderers to display your new fields.

Windows

In Resources/windows are a bunch of files that define the various screens of the app. I don’t have space in this post to go through them all, but if you look at each one, you’ll get an idea of what you need to do to make similar windows that match your project. Most of them are TableViews, which is often what you want. There’s also a MapDetailWindow that shows a static image, and other windows to render HTML content. Be sure to include any new window files you create in your Resources/app.js file. (There’s a section in it with all the includes in a list.) Also, you need to add your new windows to the app’s tab group. That can be done by following the pattern in Resources/main.js.

Connecting the App to Drupal

When I started this project, I expected to use the Services module to access the Drupal data over JSON. My plan was to create a view for each content type that I needed to download and expose those views with the Views Service module. However, when trying this approach, I came across a big limitation of Views Service. This module only exposes the Views data that is directly returned by the query. It completely skips the normal rendering step of Views. This means if you want to include the URL for a specific Imagecache-processed version of an image, it can’t be done with Views Service. You also can’t rewrite the output of fields to combine two fields into one. Pretty much everything that is awesome about Views you can’t do with Views Service.

Views Datasource to the rescue! The day after I hit this limitation, Palantir released their code for their DrupalCon app. In their post, you will see they reference the Views Datasource module as an alternative to Services. Views Datasource allows you to expose Views as JSON, and it DOES send the output through the normal Views rendering engine. So, this means you can rewrite fields, get Imagecache urls, change field labels, and probably re-theme fields if you need to, though I didn’t try this last one. With this module, I was able to get exactly the JSON output I needed to download data into the app. I’m not going to go into a full detailed explanation, because actually, if you know how to use Views, it’s pretty straightforward. It works exactly like you expect it to.

Putting Everything Together

Once I had the JSON-formatted view on the Drupal site and the schemas defined in my application, everything else was just a matter of getting of the user experience and workflow that I wanted. (Ever app is different, so you’ll have to study the Titanium API and figure out what works for your project.) After only a couple weeks of development, I was able to go from conception to launching LA2M to Go in the App Store. While it was frustrating at times deciphering both the Titanium debug output and the DrupalCon app’s design, I’m really happy with the outcome. I’ll definitely use both of these technologies in future for Drupal-connected iPhone apps, and hopefully, this post will make it a little less frustrating for you to do the same.

Useful Resources

Getting Started with Titanium Mobile Platform

How to Create a Pull-to-Refresh Table in Titanium - Everyone loves this UI control.

Drupal-iOS SDK - A separate project that connects an iPhone app to a Drupal site using only native code in XCode. Have not tried this one out.

Wayne EakerMay 10, 2011

Android Drupal Drupal Planet DrupalCon Chicago iPhone Mobile Services module Titanium mobile framework Views module

How to Import Files into Drupal CCK Fields

Wayne Eaker — Tue, 30 Nov 2010 15:28:12 +0000

How to Import Files into Drupal CCK Fields

At last night's Ann Arbor Drupal Meetup, a question was raised about how to write an import script to create Drupal nodes that include a Filefield CCK field. Filefields and Imagefields present a particular problem for import because the file must be stored into the Drupal files table before you can save them to the node. In the discussion that followed, it seemed that several of us at the meeting had developed our own ways to do this in the past for various reasons. So, I thought I would post a couple solutions for others who need to do this.

Option 1: Read the Manual

There is a manual page that explains how to programmatically import files to Filefield. However, like many Drupal manual pages, if you read through the comments, there are about 20 different versions of the code, and the discussion is often difficult for novice users to follow. The code on this page does work, but there is something simpler that we ended up with at the meeting.

Option 2: Use a Helper Function

The Filefield module has a helper function called field_file_save_file() that does a lot of the work for you. To use the function, you just need to give the original file path and a destination path if you want the file copied into a new location. Here's a simple example of a script that creates a story node and imports an image file into the "field_image" CCK field. (The file importing lines are highlighted.)

 
type = 'story';
$node->body = 'Testing.';
$node->title = 'Testing';

$file = field_file_save_file('import-images/buzz2.jpg', array(), 'sites/default/files/images');
$node->field_image = array($file);

$node->uid = 1;
$node->status = 1;
node_save($node);

This script can't be run on the command line without some other include lines to bootstrap the Drupal system. However, if you use Drush (and you should be using Drush), you don't need to do anything extra. Just run "drush scr import.php". That's it. Adding those two lines to your import script is all need to do to import files to Filefield CCK fields. Easy to do now that you know.

If you have any questions or problems using the method above, leave a comment below, and I'll see if I can help you out.

Wayne EakerNovember 30, 2010

Ann Arbor Drupal Drupal Planet Filefield module Imagefield module Import

Widespread SEO Problems with Drupal’s Taxonomy Module

Wayne Eaker — Wed, 17 Nov 2010 16:41:22 +0000

Widespread SEO Problems with Drupal’s Taxonomy Module

This week I’ve been working on fixing some duplicate content problems on our site coming from the Site Map module. In doing so, it’s become obvious that the design of Drupal’s taxonomy module makes it very easy to accidentally end up with Google indexing multiple copies of your term pages. This SEO problem isn’t just happening to Drupal rookies. Even top Drupal firms are screwing this up. Take a look at some examples from prominent Drupal company websites:

[Update: Most of these have now been fixed, so the results now are lower than I state. But they originally were correct.]

Lullabot’s Podcast Taxonomy Page
There are 84 Google-indexed pages for only 10 pager pages for this term. PLUS, there are an additional 20 indexed pages at the aliased URL for this page.

Development Seed’s “Drupal” Taxonomy Page
60 indexed pages for 31 pager pages, plus 201 indexed pages for the aliased address.

LevelTen’s SEO Taxonomy Page
19 indexed pages for 10 pager pages, plus 10 indexed pages for the alias address.

Acquia's Drupal Planet Taxonomy Page
55 indexed pages for 26 pager pages, plus 37 indexed pages indexed for the alias address.

It’s clear that Google is routinely picking up duplicates of taxonomy pages for Drupal sites. Why? Examining the results above, I see a few different reasons:

Some of these sites are not using the Global Redirect module, which means that the internal taxonomy page path “taxonomy/term/##” returns the exact same page as the aliased address. It doesn’t redirect. It just shows the same page. That in itself wouldn’t be a problem if nobody ever linked to the internal path. (because then Google would never pick it up) But, it’s clear that linking mistakes have been made at some point, or some module has exposed the internal links to Google by using them on a page. Now they are in the Google index and they won’t come out on their own.
The term depth argument for the taxonomy page (and taxonomy_term view) allows identical pages to be indexed at both at “taxonomy/term/##” and “taxonomy/term/0”, and potentially also at “taxonomy/term/##/all”. (The “/all” link may result in different content if you have hierarchical tags. But, in most cases, it’s the same.) The Global Redirect module can take care of the “/0” for you. Handling the “/all” will take some more effort. (see the solution below)
The taxonomy pages actually allow anything to be put in the depth argument position. This leads to a problem where if you have an accidental relative link in a node on the page, you will create an entirely new set of indexed pages. For example, on page two of the Development Seed results above are URLs that look like this:

https://developmentseed.org/taxonomy/term/42/www.mapbox.com?page=32

At some point in the past, there was probably a link that didn’t have an https:// at the beginning, so it was treated like a relative link. Google followed the link, and a whole new series of identical indexed pages was created by the following the pager links on these pages.

Fixing This Problem

There’s a few steps you can take on your Drupal site if you want to prevent these duplicate term pages from getting indexed or if you want to tell Google to remove already indexed pages from their results.

If you haven’t, fix your .htaccess to redirect to a single domain as described in this previous post.
Install the Global Redirect module. This will redirect the taxonomy page's internal path URL with and without the “/0” to your user-friendly alias URL.
To fix the issue with arbitrary text being appended to the URL, you can add a Rewrite rule to your .htaccess to redirect URLs with additional arguments to the main URL. The one I have put on this site is:

RewriteRule ^taxonomy/term/([0-9]+)/(.*)$ /taxonomy/term/$1 [NC,L,QSA,R=301]

This rule won’t work for sites that need to use the “/all” address, but I’m sure it can be rewritten to support that. I’m not a RewriteRule expert, though, so if someone has an alternative, please post it in the comments and I’ll update the post.
If you have hierarchical tags, and you want to make the default taxonomy page function like the “/all” page, you can enable the taxonomy_term view, remove the Term Depth argument, and then set your own depth on the Term ID argument.

Views with Pagers - Should You Index All the Pages?

The last thing you might consider doing is telling Google not to index pager subpages beyond the first one. This is a preference issue. Personally, I think it’s better to have Google focus its results on a single page for each term. In my opinion, it’s better for SEO and better for users who click on result links to go to the first page for a term rather than somewhere in the middle. But, I can understand why some people might want all the pages indexed to make sure nothing is missed.

If you do want to remove your pager pages from the Google index, you need to add a NOINDEX,FOLLOW meta tag to all the pager pages, except for the first one. There are two ways you can do this:

Change the setting for this in the Nodewords module - DO NOT DO THIS! While there is a configuration option designed for this purpose in Nodewords, it’s got a major bug (https://drupal.org/node/835172) in the stable release. Do not mess with this option, or you’ll likely end up getting NOINDEX on the exact pages you want Google to index. The thing is coded backwards or something. This issue's supposedly fixed in the -dev version if you want to try that.
Follow the instructions on this page to modify your theme page template to add the required meta tag to pager pages. This isn’t the best long-term option, especially if you’re using a standard template, since you’ll lose this change in an upgrade. But, until Nodewords is fixed, it’s the best way to go.

Conclusion

I’ve implemented all of the techniques listed above on this site, and now we have a single indexable page for each taxonomy term. All the other variations 301 redirect to the user-friendly aliased URL. Hopefully, this will allow us to concentrate as much page rank as possible on those pages for those terms. I recommend that everyone who uses the Taxonomy module on their Drupal website and is concerned about SEO take a look at how your site is being indexed. These fixes are pretty easy. But, it's clear (and surprising) that hardly anyone in the Drupal community has noticed this issue. I'm adding these items to my pre-launch checklist for all future websites.

Wayne EakerNovember 17, 2010

Drupal Drupal Planet Search Engine Optimization Taxonomy module

An SEO Problem with Drupal's Site Map Module & How to Fix It

Wayne Eaker — Tue, 16 Nov 2010 15:05:26 +0000

An SEO Problem with Drupal's Site Map Module & How to Fix It

Having recently rebuilt the Zengenuity website, it just was this week that I finally got around to setting it up on our SEOmoz account. After the initial site crawl, I was surprised to discover this:

Duplicate content in Drupal is often a problem. The URL alias system causes content to show up at multiple URLs by default. I’ve posted about this problem (and it’s solutions) before. But, I thought we doing everything right, so I was surprised to see so many duplicate content errors.

The Problem: The Site Map Module’s Default Settings

After chasing down the issue, I found that the Site Map module was the culprit. Site Map creates a simple, human-readable list of your website content using your menus items and taxonomy terms. It’s a good way to ensure that both real users and search engines know about all the content on your site. However, the default settings for the taxonomy part of the module can cause a duplicate content to show up. Here’s how:

Drupal has a built in page for each taxonomy term on your site. The URL for this page looks something like this:

https://zengenuity.com/taxonomy/term/8

If you have installed and configured the Pathauto module, this URL will probably look more user friendly. Ours looks like this:

https://zengenuity.com/blog/tags/panels-module

Theses aliases work great for normal taxonomy page links, like the ones that appear at the bottom of our blog posts. However, the aliases don't work with Site Map. Instead of using the friendly URLs by default, it appends “/all” to the original URLs. So, all the taxonomy links look like this:

https://zengenuity.com/taxonomy/term/8/all

The configuration option that controls this is here:

The reason "all" is the default is that websites with hierarchal taxonomy structures will likely want to display all content tagged with a term or any of its children. That’s what the “/all” option does. However, most sites, including this one, do not use hierarchical tags. So, this option has no effect other than to create duplicate content. When Google and other search engines index the site map page and see these "/all" links, they will index them separately from your friendly taxonomy links, and you may end up getting penalized for the content duplication.

The Solution

You should change the Site Map taxonomy setting setting to “-1” instead of “all”.

Once this is done, all the links in your Site Map page will change to their Pathauto-generated values. For sites with a flat taxonomy, there is no downside to doing this. For sites that do have hierarchical taxonomies, you can replace the standard Drupal taxonomy pages with the taxonomy_term view that comes with Views. Once you do that, you can remove the depth argument and manage the depth you want to display with the term argument.

Conclusion

Over 30,000 sites currently use the Site Map module. I’m betting that many of them have never noticed this issue before, since as site builders we rarely actually look at the site map. Luckily, this problem is pretty easy to fix, once you know it’s there. It’s just one more thing to add to your pre-launch checklist.

Wayne EakerNovember 16, 2010

Drupal Drupal Planet Search Engine Optimization SEO Site Map module Taxonomy module

Fixing Duplicate Content SEO Problems in Drupal

Wayne Eaker — Mon, 15 Nov 2010 15:50:30 +0000

Fixing Duplicate Content SEO Problems in Drupal

Having duplicate content is a major problem for search engine optimization (SEO). To put it simply, by having multiple copies of the same content online, you’re competing against yourself for search result rankings. And Google has also been explicit that duplicate content on your site may result in lower search ranking. Avoiding this penalty is important for maximizing your site's search rankings, however, Drupal's default configuration can often lead you to unknowingly create duplicate content. Read on to find out how to fix this issue.

The Problem: Drupal's URL Path System Causes Duplicate Content

In general, it’s pretty easy to avoid showing content more than one place, but Drupal users need to pay special attention to the way URL paths are managed. Let’s take a made-up example: a page called "My Puppies" on the site https://www.example.com. We'll assume that the page’s internal Drupal path is “node/10”, and we’ve given it a friendly path alias of “my-puppies”. With the default configuration of Drupal, this page will now visible at ALL of the following URLs:

https://www.example.com/node/10
https://www.example.com/node/10/
https://www.example.com/my-puppies
https://www.example.com/my-puppies/
https://www.example.com/My-Puppies
https://www.example.com/My-Puppies/

https://example.com/node/10
https://example.com/node/10/
https://example.com/my-puppies
https://example.com/my-puppies/
https://example.com/My-Puppies
https://example.com/My-Puppies/

That’s TWELVE different pages with exactly the same content, all from only one node. And it could be even more, since any variation in capitalization ("/mY-pUppIES", "/My-puppiES", etc.) will also load this page. Luckily, this problem is relatively easy to fix by doing the following two things.

Step One: Edit Drupal's .htaccess file to redirect all users to a single domain.

The .htaccess file that comes with Drupal is located at the top level of the website folder tree. (with index.php and update.php) In this file, there is a disabled section of options that looks like this:

 
# If your site can be accessed both with and without the 'www.' prefix, you
# can use one of the following settings to redirect users to your preferred
# URL, either WITH or WITHOUT the 'www.' prefix. Choose ONLY one option:
#
# To redirect all users to access the site WITH the 'www.' prefix,
# (https://example.com/... will be redirected to https://www.example.com/...)
# adapt and uncomment the following:
# RewriteCond %{HTTP_HOST} ^example\.com$ [NC]
# RewriteRule ^(.*)$ https://www.example.com/$1 [L,R=301]
#
# To redirect all users to access the site WITHOUT the 'www.' prefix,
# (https://www.example.com/... will be redirected to https://example.com/...)
# uncomment and adapt the following:
# RewriteCond %{HTTP_HOST} ^www\.example\.com$ [NC]
# RewriteRule ^(.*)$ https://example.com/$1 [L,R=301]

Pick one of the domain options by deleting the # in front of the two associated lines, and don't forget to actually put your own domain name in there. So, on this website we use:

# RewriteCond %{HTTP_HOST} ^www\.zengenuity\.com$ [NC]
# RewriteRule ^(.*)$ https://zengenuity.com/$1 [L,R=301]

You have now eliminated half of the duplicate URLs in the list above.

Step Two: Install and Enable the Global Redirect Module

The Global Redirect module does three things for you. First, it will automatically remove any slash at the end of a URL. Second, it will redirect any URL that has different capitalization to the URL address that you actually selected. So, https://www.example.com/My-Puppies will automatically redirect to https://www.example.com/my-puppies. Finally, Global Redirect will ensure that anyone accessing the original internal Drupal path (https://www.example.com/node/10) gets redirected to the path alias that you created.

Conclusion

After editing .htaccess and enabling Global Redirect, you will be left with a single URL for this content:

https://www.example.com/my-puppies

Every other variation of the URL will be redirected to this address with search engine friendly 301 redirects. By following these guidelines, you will eliminate the duplicate content that is generated by the Drupal system itself and take a big step towards maximizing the search engine ranking of your site content.

Wayne EakerNovember 15, 2010

Drupal Drupal Planet Duplicate Content Global Redirect module Search Engine Optimization

Drupal Planet

October's Most Interesting New Drupal Modules and Themes

Views Manager [sandbox]

OG Multiple Paths [sandbox]

Views Numeric Range Filter [sandbox]

Address Field Lookup [sandbox]

User Moderation [sandbox]

Mailsuite [sandbox]

September's Most Interesting New Drupal Modules

Webform Replay [sandbox]

Tableau WDC [sandbox]

Doubtfire [sandbox]

Gmail Connector [sandbox]

DrupalCamp Michigan 2015: January 31 at Henry Ford College

Decoupling Your Backend Code from Drupal (and Improving Your Life) with Wrappers Delight

But wait, there's more!

Getting Started with Wrappers Delight

Using the Wrapper Classes

Creating New Entities and Using the Setter Methods

Customizing Wrapper Classes

Drush Command Options

Packaging Wrapper Classes with Feature Modules or Other Bundle-Supplying Modules

Did You Say Something About Queries?

Wrapping Up

DrupalCamp Michigan 2013: October 12 in Ann Arbor

The Scheduled Sessions

Creating an iPhone App for a Drupal Website

The Titanium Mobile Framework

The DrupalCon Chicago App

Schema Definitions

Renderers

Windows

Connecting the App to Drupal

Putting Everything Together

Useful Resources

How to Import Files into Drupal CCK Fields

Option 1: Read the Manual

Option 2: Use a Helper Function

Widespread SEO Problems with Drupal’s Taxonomy Module

Fixing This Problem

Views with Pagers - Should You Index All the Pages?

Conclusion

An SEO Problem with Drupal's Site Map Module & How to Fix It

The Problem: The Site Map Module’s Default Settings

The Solution

Conclusion

Fixing Duplicate Content SEO Problems in Drupal

The Problem: Drupal's URL Path System Causes Duplicate Content

Step One: Edit Drupal's .htaccess file to redirect all users to a single domain.

Step Two: Install and Enable the Global Redirect Module

Conclusion