Here are a couple of examples of how exploited security holes can influence your search engine rankings.

Hacked Content

I consult for a number of clients on blogs and social media. One such client recently called me about a notice she received about “Phishing” activity on her site. It turns out that her old WordPress version (2.3.2) had been exploited and someone had added a Bank of America Phishing landing page to her site without her knowledge. We of course removed it immediately and updated her software.

If she had never received the notice, she would have gone on blissfully unaware of this hacked content on her site. If the search engines had located the Phishing landing pages, they may have deemed her site un-trustworthy and possibly even penalized her rankings because of it.

Reduction of In-Bound Links

Blogs live by sharing content and links. One of the most important ways to push your content out to other bloggers and the internet itself is via the major blog engines. Technorati.com is probably the top blog specific search engine. Technorati has recently stated they will no longer index blogs with security ridden issues.

“Because of this ongoing problem, we’re discontinuing processing crawls of blogs that exhibit common symptoms of being compromised. We strongly recommend upgrading your WordPress installation. Even if you haven’t been afflicted by a compromise, by the time you are aware that you have been a number of negative consequences may have already occurred (for instance, flagged spam by Technorati, Google or Yahoo!) — this has been reported by many WordPress users.”

Not being indexed in Technorati or the other engines will make your content much less visible to others and reduce the number of natural in-bound link you can acquire. Fewer in-bound links means lower search engine rankings.

What Software is affected?

The simple answer is “All of them”. However, to be more precise, any software which has documented security flaws that are actively being exploited by hackers.

The best thing to do is to keep your software updated to the most secure and stable version. That said, I have not upgraded to WordPress 2.5 yet as there are still a number of issues with compatibility to templates/plug-ins and as of yet, any security holes are not being exploited so far. So I can only hope the incompatibilities with be fixed before any new exploits are discovered.

Always remember, links and content are not the only things that influence your rankings.

I have added the Google Blog Bar to the homepage here to give a test. See the right hand column towards to bottom. I used 3 key phrases; “SEO”, “Social Media” and “Internet Marketing”. You can select any of these and the search results displayed will update to the current results for the phrase.

The bar has been very popular and is been seen popping up all over the blogosphere. Google is listening to customer feedback and implemented changes already. The first Blog Bar update includes changes to allow a webmaster to select a specific site to be included in the results. The site restricted search “means that you can now create a Blog Bar and program it to only search over a selected blog”.

Many SEOs are steering clear of anything “AJAX” due to its issues with crawl-ability and content accessibility. I happen to be of the camp which understands there are very good uses of AJAX from an SEO standpoint. This happens to be one of them. I think AJAX is a very good tool to “hide” content which may be considered duplicate but is still very useful for your users. This will allow the engines to focus on the unique relevant content of your pages and ignore the cannon fodder which may be spreading rapidly across the Internet. Because this content is in AJAX, I am not a bit worried that it is probably replicated on 50 or 60 other SEO blogs by now.

The Set Up:

Mr. Webmaster has a ton of very good domain names. Most are good single words which get a large amount of type in traffic. The goal here is to get these domains all pointing to the same target while preserving their and the targets search integrity. While most believe that the 301 is the standard solution in this case, there is a contingent of people who believe that the base href can accomplish the very same task.

To be clear, let’s define “search integrity” as it applies across these domains. First, only one domain can have any pages indexed. Engines do not want redirected domains in their indexes and are not very nice when they find them. Additionally, all domains need to remain penalty free. These are valuable domains and penalties would reduce their resale value. Seems simple doesn’t it …

Ok, now let’s meet the players …

Player One, Base Href:

The Base element was introduced with the HTML3.2 specification on Jan 14 1997. Not used as often as many other HTML elements, those who have embraced it have done so feverishly. The W3C HTML 3.2 Reference Specification defines the Base Element as giving “the base URL for dereferencing relative URLs, using the rules given by the URL specification”. Here’s an example of what they are talking about. In this example page we have a base element in the header and an image linked using a relative link …[html]

[/html]

The Base element will dereference the relative URL and create a fully formatted complete URL to retrieve the image as follows…[html][/html]

This is the intended function of the Base Element, however we will see in a bit that some interesting things can happen when the Base Element href points to a different domain than the one the current page is sitting on.

Player Two, 301 Re-direct:

The 301 Redirect is actually a Header Status Code implemented with the Hypertext Transfer Protocol — HTTP1.1 Specification released in Jun of 1999. The status of the HTTP 1.1 response code is “Moved Permanently”. This tells the requesting agent that the page it is requesting is no longer there and points it to a new location. Here is the W3c’s description of how it works.

The requested resource has been assigned a new permanent URI and any future references to this resource SHOULD use one of the returned URIs. Clients with link editing capabilities ought to automatically re-link references to the Request-URI to one or more of the new references returned by the server, where possible. This response is cacheable unless indicated otherwise.

There are a number of ways to implement a 301 Re-direct using both on page code and server side scripting. On the server side the most common method of 301 Re-direct is by creating a Mod-Rewrite in the the Apache based [.htaccess file]. Here is an example of a 301 redirect to control URL canonicalization:[code] # Turn on rewrite
Options +FollowSymlinks
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\.yoursite\.com [NC]
RewriteRule ^(.*)$ http://www.yoursite.com/$1 [R=301,L][/code]

Notice the “R=301″ in the code above. This tells the Mod-Rewrite to use the 301 status code when doing the re-write. You can also do Mod-Rewrite on windows based servers using ISAPI. It’s a bit more complicated but very doable.

The 301 can also be done using on page code in a variety of languages. The primary difference here is that when the re-direct is done in the page’s code, the page must be served from its current location before the new page is requested. Using a server side request, the page requested initially does not even have to exist. Here are a couple of samples of how the 301 would look to fix the same canonicalization issues done in a couple of common languages.[php] if(!stristr($_SERVER["HTTP_HOST"], 'www')){
header("HTTP/1.1 301 Moved Permanently");
header("Location: http://www.domain.com/" . $_SERVER["REQUEST_URI"]);
exit();
}
?>[/php][asp]<%@ Language=VBScript %>
<%
Response.Status="301 Moved Permanently"
Response.AddHeader "Location", "http://www.domain.com/"
%>[/asp]

Applying the Problem:

The main issues to consider when we want to determine how these two techniques maintain “search integrity” is how the search engine spiders react to and handle the requests and what happens to the pages in the index.

It has long been known that when using a 301 re-direct, search engines will only index the final URL and if the initial page has a indexing history, the page’s history will transfer to the new URL. This usually includes references from inbound links and Page Rank. Additionally, the major search engines have noted this is an acceptable technique to change URLs and do not penalize URLs using it.

In the current scenario, the parked domains will return a 301 response and transfer the user to the indicated URL using either the server script or page code methods. The 301 Re-direct will maintain “search integrity” as it is defined for this discussion.

The question now becomes “Can the Base Href do the same?” Here’s where the issue becomes cloudy. Base Href is designed to work with in a single domain. What happens when you set the Href to a domain other than to one you are on? g1smd, a member of a number of SEO forums and a participant in the discussion that prompted this article did some specific Base Href testing on this and posted his results over at the Web Master World Forums. his study showed that only using the Base Href he was able to get pages located on domain #1 indexed on domain #2 by only putting the external pointing Base href on the pages located on Domain #1. The pages never existed on domain #2 but eventually all showed in the index as belonging on Domain #2. Very interesting information indeed.

However, this test does not encompass the situation as defined in this discussion. What we have here is a number of domains with no pages which need to be pointed to a domain with pages. Pretty much the exact opposite of g1smd’s test. Let look at our criteria to meet “Search Integrity”.

Will only the final domain be indexed in the engines? Using the Base Href the pages will return a response code in the 200 range, I have seen both 200 OK and 206 Partial Content returned on pages using the Base Href. None of these responses would prevent the page from being indexed. Additionally, will the domains remain penalty free? Since the domains are serving content with no re-direct, it would depend specifically on what that content was and how it was served. The Base Href itself is not shield against a penalty at all.

Again, in the current scenario, the parked domains are indexable using only the Base Href method. Additionally, this method offers no protection to the domains from search penalties. The Base Href will not maintain “search integrity” as it is defined for this discussion.

The Conclusion:

I think it is fairly clear that the 301 Re-direct is the only way to go in this situation. While the Base Href is very useful, it does not provide the functionality or protection that would be required here to keep these domains safe and indexed properly.

What is the problem here? Well, a great piece of programming and outstanding software package does not always equal “search engine friendly”.. even if they say it does. Most programmers are just that, programmers. They do not know the ins and outs of the ever changing SEO space and if they took the time to keep up with SEO, they might fall behind on their programming skills. Most stick to what they know. Even if they do take the time to try and learn SEO, they face the same uphill battle that new SEO’s face everyday, finding reliable accurate SEO information. Too many places give bad or outdated information and how is the programmer to know the difference. This leads to great applications which are rather limited in their SEO viability out of the box. I can not think of a single online software package in any category that would not need modification to be “SEO friendly”. However, some are much worse than others and some are actually very harmful. I wont name names (yet), but I think you need to know some of the issues to look out for when evaluating software for SEO or “search engine friendliness”.

While all SEO issues apply, lets talk about the few that tend to show up more frequently in programs and scripts used to build dynamic web sites.

The first issue is Code Bloat: This is the effect of having more code in the HTML than actual text that displays on the page. The smaller your code to content ratio, the easier it is for spiders to read and understand your pages content. Here are a few things that usually add to code bloat issues.

1. On Page Script. When you view the source of a page generated by one of these programs, any scripting should be called from an external file. Javascript seems to be the most common issue, though it can happen with other scripting languages such and PHP or C++ as well.

• GOOD: All scripts called from external files
• BAD: Scripting code placed directly in the page code

2. Style Sheets. The programmers are not usually designers either and tend not to use CSS to its fullest potential. Most programmers know some basic CSS, HTML or XHTML, but this does not mean they know the best was to utilize this code for SEO. Additionally, these style sheets should be implemented as external files.

• GOOD: Designed using CSS as extensively as possible and CSS called from an external file
• BAD: Uses outdated HTML markup, places CSS in line or puts the CSS in the header of the page

3. Program Comments: Some programmers like to leave “helpful notes” in the code of a page in the form of comments. This just adds to Code Bloat. This type of information should be added to a FAQ or Doc type file.

• GOOD: No comments in the code and additional information in external documents
• BAD: Instructions or usage information placed in the pages code via comments

The next issue is Duplicate Content: This is a large SEO issue and it is growing. The engines only want one version of any specific content indexed. When they find pages that contain the same content they will pick one (seemingly at random) and list it, while any others are regulated down to Supplemental Results. Many site scripts have “features” which end up creating duplicate content issues and here are a couple of the common ones.

1. Page Templates: Many times these scripts come with dynamic pages which are built around a page template. While there is nothing wrong with this per se, these template pages can lead to duplicate content issues if done poorly. The entire code bloat issue above comes into play here. If only a small portion of the page changes dynamically, then the rest of the page is duplicated on every instance of that page. This is what leads to the duplicate content issues.

• GOOD: A large percentage of the code on each page changes dynamically
• BAD: Only small portions of the page change dynamically or design and navigation overwhelm the content

2. Session Variables: These are parameters sometimes found in the URL, which track your progress through a site. They are used for a number of purposes. They can keep track of which shopping cart info is yours or remember some information you had to input. There are ways to use sessions without this variable in the URL. If the Session ID is in the URL it causes you problems in a couple of areas. First if a few people copy the URL and link to a page, each with different Session IDs you have a duplicate content problem. Additionally, as we will cover in a bit, this adds one more variable to a URL, in which you want as few variables as possible. On a side note, the Google Webmaster Guidelines specifically states “Don’t use “&id=” as a parameter in your URLs, as we don’t include these pages in our index.” and suggests you avoid session IDs as well.

• GOOD: Does not use session variables to track surfers
• BAD: Includes session variables in the page URLs or uses “ID” as a parameter name

3. Print This Page: A wonderful feature that some of these scripts incorporate is the “print this page” feature”, It’s used here on AppliedSEO.com. However, this feature creates a duplicate version of any page it is implemented on if one small step is not taken. The most common method to fix this is to include a meta “noindex,nofollow” tag in the head of the page. Without this it is an obvious duplicate content issue.

• GOOD: The print pages are not allowed to be indexed
• BAD: These pages are allowed to be indexed

On to the Indexability issue: One of the common obstacles to good search placement is indexability. This is simply the ability of your pages to be indexed properly. In the past, this idea was attached solidly to things like “Meta Tags” and “Keyword Density”. Today we understand that many more factors go into the indexability of a web page. In an application building a dynamic website it is hard to address all of the issues, but with a focus on a few tags and content implementation you can gain some ground in search placement.

1. Dynamic Tags: While most meta tags are obsolete, there are a couple of tags you want to make sure are implemented correctly. While not a “meta” tag, the “Title” tag is the most important on the page. This tag needs to be unique on every page and be very specific to a pages’ content. The only other meta tags worth worrying about are the “description” and “keyword” tags. Each of these should also be unique and specific to the content of a page.

• GOOD: Script allows for unique Title, Description and Keyword tags on EVERY page of the site
• BAD: Tag construction automated by the software with little chance to modify or only slight modifications allowed

2. Types of Content: While this is more a function of the type of site you are building, it is important to address in some cases. Photoblogs or Gallery applications focus on images, but they need to contain a textual component. Directories or Topscripts which only do linking also need more text on the pages. It is the text on a page that is the biggest factor in indexing and if you don’t have it, the pages will not rank.

• GOOD: Each dynamic page contains a good percentage of unique topical text
• BAD: Pages only contain non-textual content or a very small percentage of unique text

3. Content Display: Making your content easy for the spiders to find is the goal here. The spiders usually read from the top of the page down, so having the main content of your page at the top of the page’s code is a good practice. Additionally, there are some neat “tricks” programmers can use to dynamically insert content into a page. Some of these tricks make the content unreadable by the spiders. If you see a scroll bar in the middle of a page for example, the content inside that element is probably not accessible to spiders. Interactive content is usually a problem from an SEO standpoint as well.

• GOOD: Clean standard HTML pages and navigation with simple a display of content
• BAD: Fancy content insertion or content that changes without reloading the page

OK, let’s talk about Spiderability: This does not have much to do with your rankings, but is more focused on allowing the spiders access to your content. If a spider has problems seeing your content or understanding what it is seeing, you will have problems getting your pages indexed at all. The search engine spiders are becoming more sophisticated every day but there are still some basic guidelines any site or page should follow, dynamically generated or not.

1. Navigation/Menus: The navigation is the key to a spider’s journey through your website. Some scripts create flashy drop down or fly-out menus that dynamically update as the site evolves. These types of menus are normally done using Javascript or DHTML both of which make the links in these menus non-spiderable. A basic textual navigation allows the spiders easier access to internal pages and additionally passes some contextual information about a page via the text used in the links. Other types of non-spiderable navigation to avoid include both javascript mouseover and image map graphical navigation.

• GOOD: Basic text link navigation
• BAD: Image based navigation, Drop down or fly-out DHTML/Javascript menus

2. Code Validation: Programmers know good code. But they don’t always know every type of code and taking shortcuts or using non-standard mark-up can cause issues with a spiders ability to analyze a page. Additionally, some enhanced features of a script may require the page to include non-valid mark-up. It is not just the page code that needs to be validated, if the page references an external CSS, it should be validated as well. By running your page through an HTML Validator or your style sheets through a CSS Validator you can easily identify coding errors and fix them. Engines have publicly stated that their spiders work better when crawling validated pages.

• GOOD: All pages and style sheets validated
• BAD: Any validation errors on pages or style sheets

3. Mod-ReWrite: I saved this for last because for some reason it seems to have become a litmus test for the SEO capabilities of site scripts. There are even mods for some scripts that only do the mod-rewrite and then claim to have made the site “Search Engine Friendly”. If you have read this far you know that could never be true. A mod-rewrite will take URLs with a bunch of parameters and turn them into normal looking URLs with keyword related file names. Since spiders have a hard time with URLs containing more than 3 parameters this make it easier for the spiders to travel the site. This is a good thing, don’t get me wrong, but it takes much more than this to make a script generated site “Search Engine Friendly”. The mod-rewrite can be a complicated task and needs to be maintained for continually updated sites.

• GOOD: An automatically maintained mod-rewrite feature
• BAD: Dynamic URLS with more than 3 parameters

It is very common for a web site generated dynamically by a script or application to have SEO issues. As a matter of fact most all of them do have SEO issues. But these types of websites are coming of age. Webmasters, SEOs and yes, Programmers are going to need to become more aware of all the different issues which can effect the “Search Engine Friendliness” of a site generated this way. Until these programs come up to SEO speed, I would not recommend using any of them without some sort of modification. Just because a script claims to be “SEO Ready” or “Search Engine Friendly” does not make it so. Compare it to the items listed above and if there are more GOODs than BADs, you have found one of the best. Most will have a majority in the BAD category. Find one which excels in the GOOD category and fix the rest, you will love the results.

Search Engine spiders read URLs very literally. So if there is any change in the URL it is considered a different page. Let’s expand on the example:

http://www.mydomain.com

http://mydomain.com

http://www.mydomain.com/index.html

http://mydomain.com/index.html

http://www.mydomain.com/index.htm

http://mydomain.com/index.htm

http://www.mydomain.com/default.asp

http://mydomain.com/default.asp

In practice, each of these URLs shows the same page. In practice, search engines see each of these as different pages since the URLs are different. So without some fixes, you now have 8 duplicate pages on your site. Now to be fair these pages wont automatically be indexed and considered duplicates, first there needs to be a link to your site or from within your site using one of the URL versions. Since you don’t know how other people might build the links to your site, you need to be proactive and prepare for the possibility that these URLs will show up.

The preferred URL for a homepage is www.mydomain.com. But you don’t have to select this as you’re preferred format. What you do need to do it to make sure every variation of the URL ends up pointing to your preference.

The first step in doing this is simple. Make sure every link on your site to the homepage uses the same URL structure. I strongly recommend using www and pointing to the root of your domain; www.mydomain.com. This way if you ever have to rename the homepage file at the root, it will not effect your current SE indexing. Renaming may happen if you switch to a dynamic site and now all your pages are now php instead of html for example.

Now about those variations. You are going to need to set up 301 redirects for those extra URLs. There are a number of ways to do redirects, but using a 301 redirect is the ONLY safe way to do it for SEO purposes. Any kind of javascript or meta redirect could get you in trouble. There are other forms of redirect as well, a 302 redirect would provide the proper function, but it is interpreted by engines as only a temporary redirect. Not good.

Implementing the 301 redirect is different depending on which web server you are using. If you are on Apache running under Unix or Linux you can add the 301 redirects via your htaccess file. If you are not sure what your htaccess file is read some of the results from this search. If you are familiar with htaccess, here is the base code you can modify to meet your needs. Just add to file and modify as needed.

# Turn on rewrite
Options +FollowSymlinks
RewriteEngine On
RewriteBase /

RewriteCond %{HTTP_HOST} .
RewriteCond %{HTTP_HOST} !^www\.yoursite\.com [NC]
RewriteRule ^(.*)$ http://www.yoursite.com/$1 [R=301,L]

If you are running IIS it’s going to be a bit more difficult. You can either use ISAPI and create the relationships yourself, or write a piece of code in PHP, ASP or .net depending on what you are using. The details of how to do these are a bit more than I had planned for this article but be sure if you Google it, plenty of people have already written how-to articles on the subject.

To sum up, you need to make sure that any URL which would normally show your homepage content is the same no matter how it is linked or what is typed in the browser. If there are variations, you leave yourself open to the possibility of have duplicate pages indexed by the search engines. Duplicate pages can be a major issue for some search engines and are something to do your best to avoid.