...

View Full Version : robots!



Sparky
04-30-2003, 08:07 PM
I am a little confused on the whole matter of robots.

How do robots come into your site design? what considerations do you take ect ect?

How is your robots.txt file sorted, and what about robot entries in your META tags?

mouse
04-30-2003, 10:28 PM
Personally, I used to run a mag that was always top on Google when I wanted it to be, I used <meta name="keywords" description=""> with a small number of key words, ie don't over do it, and <meta name="description" description=""> with a relevant description, bearing in mind someone will read it on Google/etc. I had these tags in a seperate file but they were included in the page as it was servered with php. I've not bothered directing robots to external documents holding metas.

Sparky
04-30-2003, 11:46 PM
Ohh! that wasnt quite what i was talking about, but is a lot more useful!

A couple of quick questions:-

Whats a rough guide as to how many keywords should you use? (I have 10 - is that just greedy?)

Also, my meta tags are in the following format:-

<meta name="keywords" content="stuff,morestuff,tonsoffstuff">
<meta name="description" content="description">

.....how come mine are formatted slighly different too the way you suggested, with the content="" as opposed too your description="" ?

And finnally, is seperating the meta tags and #including them just a way of keeping things tydy and easier too maintain?

Thanks a lot for your help.

brothercake
05-01-2003, 12:07 AM
Well semantically, we shouldn't really be using keywords and meta description for the purpose of improving search engine results. meta information should accurately and concisely describe the document, and no more.

And in any case - search engines are far more interested in, and give far more priority to, what the page text says - that summary you read on Google isn't the meta description, it's taken from the text.

But on a practical level - I read somewhere that you can have up to 50 keywords without worrying about it, and use the same word up to seven times.

I would advocate a middle ground - it's okay to describe the same thing in several ways - for example "dropdown menu, javascript menu, javascript navigation, dhtml menus". Think about, what will people be searching for when they're looking for something like your site? What would you search for, if you were looking?

cg9com
05-01-2003, 04:23 AM
I've seen http-equiv="keywords" before, I thought that was for HTTP headers, was that markup just wrong?
Because I've just recently seen it, I have to ask, am I missing something? :D :D :D

BrainJar
05-02-2003, 08:20 AM
The best guideline I've seen for META keywords is to use them for:

1. Describing non-text data in a page, such as an embedded video or flash presentation.

2. Providing alternate spellings for words or names used in the actual content, like "colour" for "color."

But AFAIK, almost all of the major search engines ignore them thanks to blatant abuse.

Bots are simply programs that scan the web much like people do when surfing. Different ones have different purposes. Search engines use bots to analyze add your site to their databases. Email address harvesters look for any email address they can find on a page for use by spammers. Some companies are now using bots to look for copyright violations.

Some bots look for a robots.txt file in your site's root directory. You can use it to tell the bots what directories to stay out of. Of course, scumware like those email address harvesters will ignore it altogether. But most search engine bots seem to honor it. See http://www.robotstxt.org/wc/robots.html for more info.

zoobie
05-02-2003, 10:25 AM
The best robot was "Robbie" in Forbidden Planet. Hope that helped. :D



















Seriously, meta tags have been greatly depreciated over the last few years due to abuse. Keep your keywords to an absolute minimum (around 5)...although I've seen hundreds stuffed in there. Content is now king...and the search engines will read your html and determine how relevant this all is to the search engine placement results. :cool:

Sparky
05-11-2003, 03:22 PM
Originally posted by brothercake
Well semantically, we shouldn't really be using keywords and meta description for the purpose of improving search engine results. meta information should accurately and concisely describe the document, and no more.

And in any case - search engines are far more interested in, and give far more priority to, what the page text says - that summary you read on Google isn't the meta description, it's taken from the text.



When you speak off abuse in META tags, is this referring to things like the practice of certain unscrupulous sites years ago in using words like ‘Disney’ too attract not only a lot of people too there site, but lots of people too whom the content was not appropriate?

Well, originally I posted this thread as I noticed that there were quite a lot of non-search engine bots looking through my site, and was wondering what kind of measures a webmaster/designer should take too combat/protect against this. But search engine use is just as important anyways!

I have had a good look around at some good, well designed sites, and seen their META tag use, and its pretty much cleared up its use too me.

zoobie
05-13-2003, 02:35 AM
Actually, I just came from a site that was ranked #1 on the results page in Google...even above a search engine optimization site...with absolutely no meta tags whatsoever. :D



EZ Archive Ads Plugin for vBulletin Copyright 2006 Computer Help Forum