Comments on: Scraping Google SERP

By: Bjorn Solstad (@bjornsolstad)

Bjorn Solstad (@bjornsolstad) — Tue, 15 Jan 2013 12:45:12 +0000

Joshua: The page you are hitting is the captcha page where you need to prove that you are not an automated program. What we have done to go around that, is that we are using a lot of different IP-addresses for the scraping. You will have to spread out over many different addresses in different ranges.

By: Joshua

Joshua — Thu, 14 Jun 2012 13:41:03 +0000

We can no longer get the SERP HTML from http://www.google.com/search?q=term because Google returns a 302 Moved redirect page. Any idea how to get around this?

To illustrate, hit the URL in your example with curl in term

By: noahjohn2012

noahjohn2012 — Sun, 29 Apr 2012 06:19:27 +0000

Wow! very useful tip and script.. I will try to use it now.. Thank for this..

By: Google Rank Scraping

Google Rank Scraping — Sat, 04 Feb 2012 16:20:45 +0000

For scraping Google there are a lot of additional things required, maybe you’d like to include those PHP projects: google-scraper.squabbel.com (in depth knowhow) and http://google-rank-checker.squabbel.com (contains a huge PHP project for scraping google).

Your site and articles are all really well written, I hope you find my information useful as well.

By: Bjorn Solstad

Bjorn Solstad — Fri, 09 Dec 2011 11:15:17 +0000

Ben,

What you are looking for is a link builder. There are many out there that can do this for you. We are one of them. Sen me a mail at bjorn@devenia.com if you want to know more. On our contact page you will find our phone numbers if you prefer to call. Pick the one closest to you.

By: Ben Best

Ben Best — Fri, 25 Nov 2011 06:39:18 +0000

If your not free to do it can you refer someone who can?

By: Ben Best

Ben Best — Fri, 25 Nov 2011 06:38:27 +0000

I’ll get straight to the point. I’m looking for someone who can make my website rank 1st page on Google SERPS,using my keywords.
I’ve tried whitehat seo, to find it’s a big rip off or to slow.
The major deciding factor to find someone like you is, I’m currently running a large scale Google and FaceBook PPC campaign that is being maliciously attacked by 1 or maybe 2 of my competitors, this has been proven by my site analytics, with 1,000′s of IP addresses and host names. More then likely done by a proxy server?

I’ve decide if you can’t beat them join them, now, I don’t want to attack their campaigns.

What I do want to do is find someone who can use similar techniques or better to have my site be on the 1st page of Google SERPs using my product specific keywords or phrases.
2 of my keywords have more then 16,000,000 searches a month. “Interview” & “Job Search”, as well “resume”
I’m not technical so would need someone who knows and understands how to do this.
If you can or want to do this, I will pay cash$$ for results. Time is of the essence as I’m launching a TV commercial in the next couple of days and have other great branding campaigns happening. I’m in Los Angeles California and you may contact me at 1-818-530-3259 anytime or send me your phone number and I’ll call you.

Hopefully we can establish a mutually rewarding business relationship

Regards,
Ben Best
818-530-3259

By: Dinesh

Dinesh — Tue, 23 Aug 2011 05:02:16 +0000

I need to scrap only 50 results???? Pls anyone can help me!!! I dont to search whole results its taking time to scrap…

By: Andre Loxas

Andre Loxas — Sun, 24 Jul 2011 04:55:00 +0000

But a better idea would be to scrape scroogle.org . They don’t change the html and I bet you don’t need proxies to scrape them.

But a still have no idea of WHY scrape google is for. For the mapping of keyword->sites and where to put efforts on getting a linkback?!?! Is that it?!

for the 301 redirection trick?!?

Please tell me why… I’m burning my front lobe here.

By: jan

jan — Sat, 16 Apr 2011 10:28:11 +0000

if you don’t like regular expressions to get a result you can work nicely with the DOM object in a css manner by using the ‘PHP Simple HTML DOM Parser’

http://simplehtmldom.sourceforge.net/