...

View Full Version : how does sites that do price comparison's get the info ?



umen242
03-17-2008, 02:34 PM
Hello all
how does sites that do price comparison get there info about the prices and such?
does they use the sites web services api ? or they use web spiders?

mlseim
03-17-2008, 08:32 PM
Some do "scraping" ... which isn't ethical...
Others probably subscribe to some database(s) which provide the data?

If a site offers an RSS feed, that would be an ethical way to get the data.

I would guess that the type of website (what they sell) would be
a key thing on what is available.

You can do a search on Google for "web page scraping" to see how you
can make your own spider. Like I said before, it's not really ethical to steal
bandwidth from another site without permission.

bcarl314
03-17-2008, 08:50 PM
You just sign up for multiple datafeeds and integrate them into your system. For example, if you sign up for...

Amazon.com
Buy.com
Shopzilla

You're going to have data for any product from at least 3, and probably dozens of sites.

The tricky part is setting up the database to handle that much data.

I've already integrated amazon into one of my sites (http://www.yourlistonline.com) and plan on pulling in buy.com and shopzilla in the near future.



EZ Archive Ads Plugin for vBulletin Copyright 2006 Computer Help Forum