Best approach for developing a web/database service

[gnomeontherun]

I am curious about what method people would suggest to develop a service like this:

I would like to have some form of a tracking code available for anyone to easily implement onto their site. When they update something on their site, this code would be updated so it could then be found by this service and indexed. Basically I want to be able to offer a snippet of code for sites to use, and then do daily checks for updates. So if a new blog post has been made, this snippet will be updated so the tracking could make note of it.

I am imagining a decentralized approach but I'm not a large scale programmer/developer.

I know some methods exist to do this, and it usually would be done with a type of feed. But not all information or content is passed by feeds. For instance, if a person posts a new image of a construction site occasionally, they could post a snippet and change the value with every new image for the indexing service to pick up. Lets then say there are 30 other sites doing the same thing, but aren't offering feeds or something, so they could use this snippet to get indexed and then people who really love construction images would be able to get a feed from the service of the updates.

I hope this makes sense, if not I can try to clarify. Thanks for your thoughts.

[mjlorbet]

if you're giving them code they must run, why not give them code to export a feed & just use a feed reader to incorporate the data into your site?

[gnomeontherun]

Because what if the element that I want to index is only one object that is repeatedly updated? It would be silly to have a feed with the same element over and over with only minor changes.

[mjlorbet]

change the criteria for the feed, dropping a "cookie" as you called it every time the content is updated will give you the same issue. a feed isn't an instant notification of every little change, it's just a stream of xml with given structure, the same effect as the "cookie". if you're thinking that the feed is going to produce multiple index entries for the same object, why not institute some kind of naming convention & build your index based on names? note: you'd have to do something similar with the "cookie" anyways

[oracleguy]

The RSS feed can contain whatever information you want (within spec of course), it can simply be a list of changes made and the date. Sort of like the change history on a version control software.

[gnomeontherun]

Ok here are the ideas I have so far and rationales behind using them:

RSS Feeds - Great for many things, can be designed to incorporate whatever information. Most sites already serve feeds, and can easily make new feeds.

Commenting Code Tracking - <!-- MyIndexingService("image.png") --> - In this case I just have them insert a comment with a specific convention which a parser would later pull the information from. I have no idea if anyone does this, I want to pretend like I came up with the idea :) This type of method I like because it requires only this little piece of code instead of each site serving a feed.

Pinging a site - I'm not really sure how of this works other then to PING a server for a timelapse. I thought it can be used for other things, but I don't know.

Others?

The idea is that it requires as little work on the part of the other sites I would index, and I'm not really interested in a history of updates. I would only want the most recent things.