By John Costa

 Beneath is an inventory of five reasons search engine spiders could not crawl your web site:

Flash or Java Links
robots.txt
Too Many Link on One Page
Links in Varieties
Hyperlinks in frames
Flash or Java Links

Whereas Google assures us that Flash is changing into more crawlable, in the interim, you'll probably want to avoid relying on embedding hyperlinks in Flash. In case you do it, you ought to definitely have a hyperlink exterior of Flash on the web page as well. The identical goes for Java, Silverlight, and another type of plug-in.

Robots.txt

I wrote an article a while back regarding the makes use of of robots.txt that might be useful to take a look at if you are experiencing an absence of indexing. Mainly, the robots.txt file tells the spiders that they might not access the data on a given page (ie; crawl the web page).

There will be many the cause why you wouldn't want a web page to be out there to the engines, but nonetheless need it accessible to certain individuals. For instance, you may want to enable solely certain people to view your portfolio page by means of a direct hyperlink, and never have it out there to anybody with access to Google.

Too Many Links on the Page

For what ought to be apparent causes, piling one million links on a web page is a nasty idea. For one, it is too complicated to the user. Second, it is normally an indication that you're a spammer or making an attempt to put one over on the search engines. Spiders crawl perhaps a hundred or so links on any given page. Any greater than that places you susceptible to not getting crawled.

Links in Kinds

When you have customers fill out a kind prior to viewing a web page, and that is the solely technique to access the web page, be forewarned -- it's not possible that the web page will probably be crawled. The rationale for this is, bots do not like to supply private data, nor do they wish to click on on submit buttons. Call it bigotry if you happen to like, however all the time be positive to embody a hyperlink to your content that does not require a form to be submitted.

Hyperlinks in Frames

Generally if you use frames on your page, Google, Yahoo, and Bing get confused by the structure of your website, and that confusion can result in a site visitors jam. Besides ruining your web optimization, frames, to me, are simply hideous anyway. Avoid using them to avoid this common pitfall.

John Costa: Author, Speaker, Coach, Social Media Marketer Seo, Marketing and Product Review