Hello and welcome to our community! Is this your first visit?
Register
Enjoy an ad free experience by logging in. Not a member yet? Register.
Results 1 to 5 of 5
  1. #1
    New Coder
    Join Date
    Jul 2012
    Posts
    73
    Thanks
    4
    Thanked 0 Times in 0 Posts

    Simple Scraping Project

    1) Project Details: (be as specific as possible):

    I am an avid traveler. I currently have a site I created that is fairly simple. I put in my flight data and the url is created for about 80 websites. I can then navigate to them individually, in small groups or all at once. Each request is handled in a new tab, window.open through JS..

    I'm already calculating the URL, so that part is done.

    What i would like to do, is figure out a way to stop having to open up 80 tabs for a search.
    I would like to select the engines I want to search (or select all), and have the page return the best result from each url, sorting with the best deal at the top.

    Ideally someone would install the correct libraries on my machine
    once that's complete, I believe a single dynamic function would do the trick.
    the function would load the calculated url, and a fairly long ifelse statement to determine which url is loaded. then based on the url loaded it would determine the xpath or whatever data needed (based on my or your research for each engine, i'm flexible here).

    The site is hosted on my vps, and currently has:
    php 5.3
    phantomjs
    node.js
    node.io
    yql
    python 2.4
    python 2.7
    perl 8.16
    (can't think of anything else that may be helpful. feel free to ask for specifics and I can reply)

    2) Payment method/ details (Paypal, check? Timeline?):

    paypal is fine.. open on this..
    timeline.. i have no timeline.. what i have now works just fine.. but I guess I'm more lazy than I thought

  • #2
    Regular Coder Redcoder's Avatar
    Join Date
    May 2012
    Location
    /dev/null
    Posts
    333
    Thanks
    2
    Thanked 47 Times in 46 Posts
    Hi steve...PM me or email me at william.thedev[at]gmail.com for this. I have done a lot of web scraping projects so I'm a good fit for this.

  • #3
    New Coder
    Join Date
    Jul 2012
    Posts
    73
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Hi red, sorry, I didnt see this message, I thought everything would be coming to PM

    I'd be interested in chatting about future projects though

  • #4
    New Coder
    Join Date
    Jul 2012
    Posts
    73
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Project awarded

  • #5
    New Coder
    Join Date
    Jul 2012
    Posts
    73
    Thanks
    4
    Thanked 0 Times in 0 Posts
    Project was awarded to member: ndsdb

    Estimated time frame = 48 hours.
    Paid 1/2 up front. I have not heard from him since.

    FYI, don't work with him. I will post if he decides to complete the project as agreed.


  •  

    Posting Permissions

    • You may not post new threads
    • You may not post replies
    • You may not post attachments
    • You may not edit your posts
    •