View Full Version : Help: How to read and store the input stream of data from a URL?

07-03-2007, 05:22 PM
Hello everyone,
I was wondering if anyone could help me please. I want to read the html code from a url and store it into a string, but I don't know how to go about it. Any help would be greatly appreciated.

So far what I have done is this function, which loads the Google search results on a page. I have a lot of other code that helps in loading the Google search page from a text box. My aim is to retrieve all the html code after doing a google search and store it in a long string (adding each line, one after another).

function search(){
var url = "";
var searchTermsBox = document.getElementById("input");
URL = "http://www.google.com/search?q=" + searchTerms;

I tried using xmlhttprequest() object, but am not sure how to use it.

var req = new XMLHttpRequest();
req.open('GET', url, true);
req.onreadystatechange = function (aEvt) {
if (req.readyState == 4) {
if(req.status == 200)
dump("Error loading page\n");
alert (req);
var resultsgroup = req;
//var resultsgroup = document.getElementById ("res");
alert ("resultsgroup"+resultsgroup);

basically what I want to do is just retrieve the html source code from a web page and store it in a string. I could use the stream reader to concatenate and store the stream as a string. Not sure how to go about it though.
I realise all the code above may be confusing, but you can ignore it if you want and suggest some ideas.
Thank you

07-03-2007, 06:20 PM
And where do you propose to store this string? Using JS, unless you are using AJAX and feeding that string to a server script, it can ONLY store on the client end. What you need is a server side solution (ColdFusion, ASP, PHP, etc.). The ONLY client side storage is via a cookie and that is limited by the browser as to it's size (IIRC, it's very small ... like 4KB), which will not store much code.

07-04-2007, 03:24 PM
Thanks for replying FastCougar,

I just wanted to rephrase the question. what I want to do is store the html source code in a string variable so that I can strip all the html tags and get the retrieved Google search results.

What I am doing is building a firefox extension. I am using javascript to retrieve the Google search results.

So basically when I do a Google search, I want to retrieve the html code instead of what is seen in the screen.

Any ideas how I can do this?

Thanks again.

07-04-2007, 06:08 PM
I guess not knowing what you want to do after is what's mixing me up here.
Have you been to code.google.com?
Chances are they have what you're looking for or a nice starting point to what you're looking for there