Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 9 of 9
  1. #1
    New Coder
    Join Date
    Jul 2005
    Posts
    92
    Thanks
    4
    Thanked 0 Times in 0 Posts

    Scrolling through urls to download files

    There's a browser based game I play where you can scan sections of a grid and download an xml file of the scan. I'm trying to automate this to speed things up.

    It's easy enough to generate the urls for each section of grid to download but the xml file has to be generated.

    How do I actually download each xml file in the process of looping through each coordinate?

    Here's what I tried so far that only resulted in the last coordinate being process (the URL has obviously been changed):

    PHP Code:
        function scanSystem() {
        
    // Starting value for scans
            
    var xMin '0';
            var 
    yMin '0';
        
    // Temporary - The highest coordinates that will be scanned
            
    var coordMax '5';

            for (var 
    xMin<= coordMaxx++) {
                for (var 
    yMin<= coordMaxy++) {
                    var 
    url "http://xmlurl.com/index.php?x=" "&y="y;
                    
    window.location url;
                }
            }
        } 
    Last edited by Styles2304; 02-17-2013 at 02:51 AM.

  • #2
    New Coder
    Join Date
    Jul 2005
    Posts
    92
    Thanks
    4
    Thanked 0 Times in 0 Posts
    I tried modifying it slightly by creating an iframe and changing the src with a delay but all that does now is display the last xml file. Doesn't display any others and no longer downloads it:

    PHP Code:
        function scanSystem() {
        
    // Starting value for scans
            
    var xMin '0';
            var 
    yMin '0';
        
    // Temporary - The highest coordinates that will be scanned
            
    var coordMax '2';

            for (var 
    xMin<= coordMaxx++) {
                for (var 
    yMin<= coordMaxy++) {
                    var 
    url ""http://xmlurl.com/index.php?x=" + x + "&y="+ y;
                    
    document.getElementById('iframe').src url;
                    
    sleep(2000);
                }
            }
        }

        function 
    sleep(milliSeconds){
            var 
    startTime = new Date().getTime(); // get the current time
            
    while (new Date().getTime() < startTime milliSeconds); // hog cpu
        


  • #3
    Senior Coder
    Join Date
    Jan 2011
    Location
    Missouri
    Posts
    3,763
    Thanks
    23
    Thanked 548 Times in 547 Posts
    Some thoughts on the first post.
    You don't need to make this a function do you? Just saying this could run automatically I think, You know better then me about this.

    The window.location executes every time the loop runs from x=0 y=0 to count of 5 each. I am not sure if this is not what you want.

    The question is How will you control where the loops stop and retrieve the url.
    Last edited by sunfighter; 02-16-2013 at 04:49 PM.

  • #4
    New Coder
    Join Date
    Jul 2005
    Posts
    92
    Thanks
    4
    Thanked 0 Times in 0 Posts
    It's only a function so I can call it on click . . . it will also be customizable in the future to allow for scanning only certain sectors.

    As far as the loop stopping and retrieving the URL . . . that's the trick. I want it to retrieve the url at each step. At each step it generates the url for a dynamically generated XML file that I need to download.

    Like I said though, it's a grid so it needs to download the url file for every from (0,0) to (2,2). (I shortened the loop for sake of testing)

    The loop generates those coordinates but it doesn't access any file accept the last one.

  • #5
    Senior Coder rnd me's Avatar
    Join Date
    Jun 2007
    Location
    Urbana
    Posts
    4,184
    Thanks
    10
    Thanked 569 Times in 550 Posts
    here is some code to download a local URL.
    instead of the iframe, feed to url to the dlURL() function below.

    this works best in chrome, where a single click can authorize "downloading multiple files".

    the function will download the xml using ajax, and then use HTML5 to download the result to your local machine's downloads folder.

    don't even ask about trying to save them in a specific folder, we're lucky enough that even this works...


    Code:
    function dlURL(url){
      download( IO(url), url.split("?")./slice(-1)[0].replace(/\W+/g,"_")+".xml", "text/xml" );
    }
    
    function IO(U){
      var X=new XMLHttpRequest;
      X.open("GET",U,false);
      X.send();
      return X.responseText;
    }
    
    function download(strData, strFileName, strMimeType) {
    	var D = document,
    		A = arguments,
    		a = D.createElement("a"),
    		d = A[0],
    		n = A[1],
    		t = A[2] || "text/plain";
    
    	//build download link:
    	a.href = "data:" + strMimeType + "," + escape(strData);
    
    
    	if (window.MSBlobBuilder) {
    		var bb = new MSBlobBuilder();
    		bb.append(strData);
    		return navigator.msSaveBlob(bb, strFileName);
    	} /* end if(window.MSBlobBuilder) */
    
    
    
    	if ('download' in a) {
    		a.setAttribute("download", n);
    		a.innerHTML = "downloading...";
    		D.body.appendChild(a);
    		setTimeout(function() {
    			var e = D.createEvent("MouseEvents");
    			e.initMouseEvent("click", true, false, window, 0, 0, 0, 0, 0, false, false, false, false, 0, null);
    			a.dispatchEvent(e);
    			D.body.removeChild(a);
    		}, 66);
    		return true;
    	} /* end if('download' in a) */
    	; //end if a[download]?
    
    	//do iframe dataURL download:
    	var f = D.createElement("iframe");
    	D.body.appendChild(f);
    	f.src = "data:" + (A[2] ? A[2] : "application/octet-stream") + (window.btoa ? ";base64" : "") + "," + (window.btoa ? window.btoa : escape)(strData);
    	setTimeout(function() {
    		D.body.removeChild(f);
    	}, 333);
    	return true;
    } /* end download() */
    my site (updated 13/9/26)
    BROWSER STATS [% share] (2014/1/19) IE7:0.2, IE8:6.7, IE11:7.4, IE9:3.8, IE10:4.4, FF:18.3, CH:43.6, SF:7.8, MOBILE:27.5

  • #6
    New Coder
    Join Date
    Jul 2005
    Posts
    92
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Ok outstanding . . . the only issue is the dlURL function is breaking the code. I commented out line by line until I found it but I don't know regEx enough to fix it. See anything odd?
    Last edited by Styles2304; 02-16-2013 at 09:05 PM.

  • #7
    Senior Coder rnd me's Avatar
    Join Date
    Jun 2007
    Location
    Urbana
    Posts
    4,184
    Thanks
    10
    Thanked 569 Times in 550 Posts
    oops, pesky tiny keyboards:


    Code:
    function dlURL(url){
      download( IO(url), url.split("?").slice(-1)[0].replace(/\W+/g,"_")+".xml", "text/xml" );
    }
    my site (updated 13/9/26)
    BROWSER STATS [% share] (2014/1/19) IE7:0.2, IE8:6.7, IE11:7.4, IE9:3.8, IE10:4.4, FF:18.3, CH:43.6, SF:7.8, MOBILE:27.5

  • #8
    New Coder
    Join Date
    Jul 2005
    Posts
    92
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Don't know. I think there's still something wrong with the dlURL . . . the alert right after it's called never fires:

    PHP Code:
    // Functions to setup
        
    function scanSystem() {
        
    // Starting value for scans
            
    var xMin '0';
            var 
    yMin '0';
        
    // Temporary - The highest coordinates that will be scanned
            
    var coordMax '2';

            for (var 
    xMin<= coordMaxx++) {
                for (var 
    yMin<= coordMaxy++) {
                    var 
    url "http://xmlurl/members/scanners/list.php?cockpit&xml&x=" "&y="y;
                    
    dlURL(url);
                    
    alert('success');
                }
            }
        }

        function 
    dlURL(url){
            
    downloadIO(url), url.split("?").slice(-1)[0].replace(/W+/g,"_")+".xml""text/xml" );
        }

        function 
    IO(U){
          var 
    X=new XMLHttpRequest;
          
    X.open("GET",U,false);
          
    X.send();
          return 
    X.responseText;
        }

        function 
    download(strDatastrFileNamestrMimeType) {
            var 
    document,
                
    arguments,
                
    D.createElement("a"),
                
    A[0],
                
    A[1],
                
    A[2] || "text/plain";

            
    //build download link:
            
    a.href "data:" strMimeType "," escape(strData);


            if (
    window.MSBlobBuilder) {
                var 
    bb = new MSBlobBuilder();
                
    bb.append(strData);
                return 
    navigator.msSaveBlob(bbstrFileName);
            }



            if (
    'download' in a) {
                
    a.setAttribute("download"n);
                
    a.innerHTML "downloading...";
                
    D.body.appendChild(a);
                
    setTimeout(function() {
                    var 
    D.createEvent("MouseEvents");
                    
    e.initMouseEvent("click"truefalsewindow00000falsefalsefalsefalse0null);
                    
    a.dispatchEvent(e);
                    
    D.body.removeChild(a);
                }, 
    66);
                return 
    true;
            }

            
    //do iframe dataURL download:
            
    var D.createElement("iframe");
            
    D.body.appendChild(f);
            
    f.src "data:" + (A[2] ? A[2] : "application/octet-stream") + (window.btoa ";base64" "") + "," + (window.btoa window.btoa escape)(strData);
            
    setTimeout(function() {
                
    D.body.removeChild(f);
            }, 
    333);
            return 
    true;
        } 

  • #9
    New Coder
    Join Date
    Jul 2005
    Posts
    92
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Here's the solution given me by Joseph the Dreamer on stackoverflow. I've modified it slightly but overall, this is what's working for me:

    PHP Code:
    function scanSystem() {
        var 
    x,
            
    y,
            
    xMin 0,
            
    yMin 0,
            
    coordMax 19,
            
    url,
            
    iframeContainer document.getElementById('iframe_container');

        for (
    xMin<= coordMaxx++) {
            for (
    yMin<= coordMaxy++) {
                
    url 'http://www.xmlurl.com/members/scanners/list.php?cockpit&xml&x=' '&y=' y;
                $(
    '<iframe/>',{src url}).appendTo(iframeContainer);
            }
        }

    What it does is dynamically generate an iframe for each url and then load it in it's own time. In doing this, it downloads each file as the content in the iframe loads. Works flawlessly. iframeContainer is just a hidden div so all of this happens behind the scenes. Hopes this helps somebody down the road.


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •