Do search engines index fly pages?

[bazz]

Hi,

I have decided that as my, still increasing web site has already over 6000 files and is about 116Mb in size, It is time to utilise more Internet capabilites.

I am replacing hard pages with .pl created fly pages so that at the end up, I should have a couple of stylesheets and four cgi created pages for all of my content.

What I would like to know is, how much of an impact on potential visitors, will search engines make, where my pages are not hard coded, but created on the fly. Will the engines (esp Google), index it properly?


thanks.

[Mhtml]

Google can only see the end result of server side stuff which will be a hard coded page basically.

[edit:] Not just google, anything.

[Nightfire]

From what I read a few months back at work was that Google has upgraded their software and can index sites with query strings and stuff in the urls, but not as good as a hard coded page. Now if this is true or not, I have no idea as it doesn't index my sites, but it does index pages in forums, so i'm kinda lost how it works

[brothercake]

I don't think it's technical so much as policy - if you fully index a dynamic site then you may end up indexing loads of pages which are only slightly different from each other.

But clearly Google does index them ... but perhaps with restrictions or some kind of conditional logic ...

Who knows ... the way past it is to use the "clean URIs" technique of disguising CGI parameters to look like normal folders - eg "domain.com/something/nothing" could translate to "domain.com?a=something&b=nothing" using mod_rewrite in Apache (among other techniques), which is easier for users to remember and type, as well as getting past the ambiguity of whether robots will index the pages.

See this article for more info and speculation http://www.evolt.org/article/Making_clean_URLs_with_Apache_and_PHP/18/22880/index.html

[bazz]

Thanks guys.

Originally posted by brothercake
I don't think it's technical so much as policy - if you fully index a dynamic site then you may end up indexing loads of pages which are only slightly different from each other.

Mmm. My pages are all different in appearance and information but the only difference in the constructuion of the pages is the source of the text graphics and stuff.

Who knows ... the way past it is to use the "clean URIs" technique of disguising CGI parameters to look like normal folders - eg "domain.com/something/nothing" could translate to "domain.com?a=something&b=nothing" using mod_rewrite in Apache (among other techniques), which is easier for users to remember and type, as well as getting past the ambiguity of whether robots will index the pages.

I dont fully understand you there brothercake. My url is along the lines of
ww.mydomain.com/cgi-bin/accommodation.pl?dpt=deptname_dpt

I'll check out the article you suggested.

thanks :thumbsup:

[knightcon]

Search engine spiders work much the same as human sight in that they don't "see" the HTML tags, only the text, as a general rule of thumb if you want to know what the search engine spider will see get a text only browser and enter the page URL, what is displayed is what the search engine spider will see.

Additionally as for query strings it is true it is more policy rather than function as the full URL is passed by the search engine spider when it logs a URL for searching. The trick with anything like this is that the search engine spider will still check it but if it is not significantly different to other such pages it has indexed it won't index it. As a general rule of thumb with query strings, make sure the page title is different every time, make it a page heading not a site heading, and make sure you have meta tags in place for keywords and page description and the keywords and description are significantly enough different that the search engine spider will realize the page is a different page.

Also look into apache mod_rewrite, you can make any URL you want and then in your .htaccess file you can create rules which translate what the URL says into a query string for your page.

EG...

http://www.example.com/accomodation/dept_name

will be converted into

www.mydomain.com/cgi-bin/accommodation.pl?dpt=deptname_dpt

by mod_rewrite.

[bazz]

speedy reply. thanks anyway. :D

[teedoff]

speedy reply. thanks anyway. :D

lol funny... 8 year old post!