Simple Scraping Project
1) Project Details: (be as specific as possible):
I am an avid traveler. I currently have a site I created that is fairly simple. I put in my flight data and the url is created for about 80 websites. I can then navigate to them individually, in small groups or all at once. Each request is handled in a new tab, window.open through JS..
I'm already calculating the URL, so that part is done.
What i would like to do, is figure out a way to stop having to open up 80 tabs for a search.
I would like to select the engines I want to search (or select all), and have the page return the best result from each url, sorting with the best deal at the top.
Ideally someone would install the correct libraries on my machine
once that's complete, I believe a single dynamic function would do the trick.
the function would load the calculated url, and a fairly long ifelse statement to determine which url is loaded. then based on the url loaded it would determine the xpath or whatever data needed (based on my or your research for each engine, i'm flexible here).
The site is hosted on my vps, and currently has:
(can't think of anything else that may be helpful. feel free to ask for specifics and I can reply)
2) Payment method/ details (Paypal, check? Timeline?):
paypal is fine.. open on this..
timeline.. i have no timeline.. what i have now works just fine.. but I guess I'm more lazy than I thought