Comments on: Scraping Google SERP

By: Louie Sison

Louie Sison — Tue, 09 Feb 2010 08:56:51 +0000

Search Engine Optimization is a passion. You got it right in this article

By: seozero

seozero — Wed, 16 Dec 2009 07:50:48 +0000

anon, plz look here
http://php.net/manual/en/reference.pcre.pattern.modifiers.php

Try to experiment if you can’t get it. Remove/add modifiers one by one and see what happens.

HTH.

By: anon

anon — Tue, 15 Dec 2009 14:43:24 +0000

In your preg_match_all, could you explain what the siU bit at the end does?

By: google scraper

google scraper — Fri, 20 Nov 2009 03:55:55 +0000

First:
Thanks for the article, it’s interesting.

I found another one specialized on google scraping:
http://google-scraper.squabbel.com

If you don’t allow other URLs here just remove it please.
But that one goes into much more detail and it has a much more advanced PHP project included.

It can filter advertisements and it scrapes all google pages. (It mainly aims for large scaled scraping)

Well thanks again for your nice site

By: proxy guy

proxy guy — Tue, 18 Aug 2009 21:52:53 +0000

Hey Brad!
You probably tried to use “public proxies”.
Those are a waste of time.
Most are incredible slow, overused and short lived.

If you want good results and a reliable system you’ll have to invest a few bucks.
I’ve been using Cloakfish (www.cloakfish.com) for a while.
Cloakfish is cheap, you can scrape a lot for small bucks.
The downside is lower performance.

Check the site I linked, that’s seo-proxies.com (www.seo-proxies.com). They are specialized on scraping and similar jobs and offer php scripts that do most of the job (you only have to add the code from this blog and you’re done)

hope that helps

By: Brad

Brad — Tue, 18 Aug 2009 02:10:59 +0000

I feel lame for asking…but do you have any recommendations as to where to get proxies. All the ones I got with a google search come back with ‘couldn’t connect to host’ (and I’ve tried about 50 so far)

By: seozero

seozero — Thu, 13 Aug 2009 16:57:01 +0000

Valuable info, thanks a lot!

By: proxies for scraping

proxies for scraping — Thu, 13 Aug 2009 08:26:38 +0000

Just as info, you do not need sleeps or similar.
The trick is to change the IP for each new search term, you can browse all sites of a keyword (max 10 sites/1000 hits in google) without changing ip and risking a ban.
But as soon as you change the keyword you need to change your IP as well.
If you don’t do that google will add a captcha before you can continue.
If you continue google will temporarily block the IP for a few hours.
If you continue again it can happen that you go into a permanent block, that’s something you want to avoid at all cost.

greets

By: seozero

seozero — Sun, 12 Jul 2009 17:46:12 +0000

Jan, copy print_r($result); after $result = getPage… See what data you get.

And it’s always a good idea to try another proxy.

The code works, trust me ;] Have a nice day

By: Jan Alvin

Jan Alvin — Sun, 12 Jul 2009 10:25:55 +0000

Hi! I find your script to be VERY useful and best of all its free

but I can’t seem to make it work. It only shows a blank page. (I’ve already placed the code for the getpage function.)

please help me.