Comments on: Scraping websites with PHP cURL under proxy

By: gonzalo

gonzalo — Fri, 12 Mar 2010 01:09:18 +0000

couldn’t connect to host

$result = getPage(
‘li56-17.members.linode.com:3128′, // use valid proxy
‘http://www.google.com/search?q=twitter’,
‘http://www.google.com/’,
‘Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8′,
1,
5);

Thanks

By: JJ Rohrer

JJ Rohrer — Wed, 25 Nov 2009 16:50:59 +0000

There are some links about the legality of scraping that i’ve been putting together here: http://www.legalcasual.com/questions/1/automated-site-scraping

By: Ar

Ar — Tue, 24 Nov 2009 07:22:21 +0000

Hi there.. thx for useful info.. I’m a noob actually and this helped me alot.. but i have one problem.. I need to round-robin my proxies how can I implement that?
any help is very appreciated..

By: seozero

seozero — Wed, 28 Oct 2009 07:18:25 +0000

@Berto, are you picking connect/forward proxies? Or HTTP?

By: Berto

Berto — Tue, 27 Oct 2009 05:55:31 +0000

I am trying this and keep getting Proxy CONNECT aborted. I believe it’s error 56. Do you know what that means? Googling around for it has been useless, and I know the proxy is good as I just used a proxy checker and am randomly choosing many proxies.

Thanks!

By: seozero

seozero — Fri, 16 Oct 2009 07:37:16 +0000

@Tony, try to find some alive proxies. Double check all params. BTW, what are you scraping? Is it server or client side scraper?

By: Tony

Tony — Thu, 15 Oct 2009 07:49:55 +0000

Thanks, this has been very helpful…
except for some reason I can’t get *any* proxy to work.
Every time I just get curl error: couldn’t connect to host. No further info is provided and it’s frustrating,
so I thought I’d vent here. ARRGGHH!!!!! Thanks.

By: seoproxies.com

seoproxies.com — Tue, 18 Aug 2009 17:16:31 +0000

I’ve signed up for a free trial on seo-proxies.com.
Can you give me an example how to use the API now ?

I want it to automatic change my IP and scrape google for a keyword in specific languages.

thanks !

By: US ips

US ips — Thu, 13 Aug 2009 08:16:46 +0000

Hi,
I’m using the proxies at seo-proxies.com for web scraping.
The great things: they have a php API and php source code examples !

So you don’t have to think about proxy user authentication (usernames passwords) or how to change and manage your proxies ..
All you do is take their free php source code, add your userid and password into it, and then add a few lines to specify what to do.
The proxy is working from begin on, changing the proxy IP is just a single php line.

It’s really great how easy programming web tools can be!

greets,
Henk

By: Val

Val — Tue, 23 Jun 2009 03:49:36 +0000

What if I have proxies w/ login?