...

View Full Version : Why is my site not crawled?



coolguythampy
09-21-2009, 05:24 AM
Hello,

I have a wellness blog (http://healthruns.com) which isn't getting crawled. The cache is updated by Google on last September 13th. Why is Google not updating my site in their database?
I have built links after that. I have links pointing to my internal pages. i have site maps submitted.

What am I doing wrong? Also how do I stop Google from indexing my plugin pages.

mlseim
09-21-2009, 01:22 PM
Your site as it appears here:
http://www.google.com/#hl=en&source=hp&q=health+tips+and+fitness+resource&aq=f&aqi=&oq=&fp=ee36edbd3c16a1c5

Ranked 4 out of 17 million.

It says it was last cached Sept 13, 2009 (click the cache link).
That's only 1 week ago.

What exactly do you mean by "isnt' getting crawled"?

Also ... the title of your site: Health Tips and Fitness Resource
is so generic, and the content may not fit the title. Google likes
to have content that pertains to what your site is about, the title,
description, etc. Right now, it's all about no smoking, but in a week,
it might be something else. That's difficult to cache and rank.

Get Google Analytics and also view your webhost web stats log.

coolguythampy
09-21-2009, 01:33 PM
It says it was last cached Sept 13, 2009 (click the cache link).
That's only 1 week ago.

What exactly do you mean by "isnt' getting crawled"?


That's the problem. I have posted around 7 or more articles after the last crawl. Those contents are not getting indexed. I havent had such an experience. Normally all my blog posts, once the main page gets indexed, are indexed within 1-3 hours of posting. This seems too long (1 week).


Right now, it's all about no smoking, but in a week,
it might be something else. That's difficult to cache and rank.

So do you suggest I set up a static front page for my blog? Well isn't the problem you mentioned normal with all blogs?


Get Google Analytics and also view your webhost web stats log.

I already have Google analytic installed.

seco
09-21-2009, 02:10 PM
get the all in one seo pack plugin for wordpress

coolguythampy
09-21-2009, 02:12 PM
get the all in one seo pack plugin for wordpress

I am using thesis theme. So there is no use of all in one SEO

mlseim
09-21-2009, 07:42 PM
So you're saying that the ranking of #4 out of 17 million isn't an issue ...

You expect that I should be able to type in "bodybuilding techniques" and
instantly see the post you've made today, at a top 10 ranking? On Google?

Describe why your site should show "bodybuilding techniques" in the top 10
within 3 hours of posting. I think it's so unrealistic.

Am I not understanding what "indexed" means?
Does it relate to ranking, appearance on Google, or an Adsense kind of thing?

coolguythampy
09-21-2009, 07:45 PM
I think you are not getting the point. let me be clear. i am not talking about top 10 rankings. I am saying my posts are not getting indexed by Google. that means not appearing in googles database.

Check using site:healthruns.com You can see some pages indexed but the latest posts not getting indexed or coming in that list.

masterofollies
09-21-2009, 08:00 PM
It's nearly impossible I believe for a database as big as googles to update nearly instantly. It's just like changing nameservers, can take up to 72 hours to go across the world and have everyone tuned in to the new one.

A week isn't bad, and being #4. I'd be ever so happy.

mlseim
09-21-2009, 08:44 PM
I guess that the fact that "it used to", but "now it's not" is probably more of
a Google server issue than anything with your site. You've really changed
nothing since it stopped indexing. Perhaps Google has changed some of its ways?

EDIT:
Interesting ...
http://www.google.com/#hl=en&source=hp&q=healthruns.com&aq=f&aqi=&oq=&fp=ee36edbd3c16a1c5

Look at most of the posts.
The first 3 Google pages (hits) are all about rankings, searches, appraisals, etc.
Maybe the content should be more important than the "indexing".

I would expect the first 3 Google pages (hits) to be about my health and fitness.
Why aren't they? Isn't that the point of having the blog in the first place?

You have some issues with "site ego".


.

coolguythampy
09-21-2009, 10:19 PM
I am not complaining that I am not ranking first for the term "Health" or anything. i was just wondering what happened that my site is not getting crawled after 13th? My other sites are getting crawled. only if Google crawls my site can they index new contents right? I haven't made any changes yet. I am going to get a coder to modify my theme

BTW- By The Way!!!

The first 3 pages cannot be filled because Google is not crawling my site to get new contents fetched ot their database. So my site is still with small number of articles

seco
09-21-2009, 10:21 PM
I am using thesis theme. So there is no use of all in one SEO

what does a theme have to do with seo? get google sitemap plugin then

coolguythampy
09-21-2009, 10:49 PM
what does a theme have to do with seo? get google sitemap plugin then

Thesis is SEO optimized. I have fields in the theme itself to put in title, description, keyword etc. Also options which all in one SEO provides and more

seco
09-21-2009, 11:55 PM
Thesis is SEO optimized. I have fields in the theme itself to put in title, description, keyword etc. Also options which all in one SEO provides and more

does the THEME create a sitemap and ping search engines? Im sure it doesnt, you setup a google webmaster tools account to monitor your sites seo stats?

coolguythampy
09-22-2009, 02:04 AM
does the THEME create a sitemap and ping search engines? Im sure it doesnt, you setup a google webmaster tools account to monitor your sites seo stats?

Did everything you mentioned at the start of site creation itself

seco
09-22-2009, 02:38 AM
ok so it includes all those PLUGINS? you need to make sure every post you make gets added to the sitemap.xml "not rss" and that sitemap.xml is submitted to google.

coolguythampy
09-22-2009, 02:57 AM
ok so it includes all those PLUGINS? you need to make sure every post you make gets added to the sitemap.xml "not rss" and that sitemap.xml is submitted to google.

No. I installed separate plugins for that. Every post is added to the sitemap.xml file

http://healthruns.com/sitemap.xml

seco
09-22-2009, 03:06 AM
ok good, is it submitted to google webmaster tools?

coolguythampy
09-22-2009, 03:32 AM
ok good, is it submitted to google webmaster tools?

Yes. It says last downloaded on 21st this month.

Sitemap stats
Total URLs: 50
Indexed URLs: 18

Format is ok.

There are two sitemaps

sitemap.xml and sitemap.xml.gz. is that ok?

seco
09-22-2009, 03:37 AM
yeah, just wait.

coolguythampy
09-22-2009, 12:07 PM
yeah, just wait.

I guess I have to wait till everything stabilizes.

seco
09-22-2009, 01:06 PM
I guess I have to wait till everything stabilizes.

take into account with google, domain age and domain registration time. the younger it is "under a year" it takes google some time to trust you, a longer domain registration also helps.

cher
09-22-2009, 02:07 PM
There are two sitemaps

sitemap.xml and sitemap.xml.gz. is that ok?

No problem. You can have that if the sitemap is bigger is size.

coolguythampy
09-22-2009, 07:04 PM
How do I stop Google from indexing my plugin files? They are indexing the wrong documents. Why index the plugin files when there are lots of juicy contents ;) How do I configure the robots.txt file to do that? Will the following work?



User-agent: *
Disallow:
Disallow: /cgi-bin/
Disallow: /wp-content/plugins/
Sitemap: http://healthruns.com/sitemap.xml

coolguythampy
09-26-2009, 01:49 PM
Is the above format correct?

coolguythampy
09-26-2009, 04:17 PM
Is there moderators in this forum?

Update: Great. this guy is banned now



EZ Archive Ads Plugin for vBulletin Copyright 2006 Computer Help Forum