All you need is a web service and a plugin. The more blogs are “plugged”, the more domains and links will be saved.

Domain Collector Web Service

The purpose is obvious – save domains to database. You can code smth like that:

<?php
if (isset($_POST['domain'])) {
    $mysql_connect = mysql_connect('localhost', 'login', 'password');
 
    if ($mysql_connect) {
        mysql_select_db('database', $mysql_connect);
 
        $domain = strtolower(mysql_real_escape_string($_POST['domain']));
        $res = mysql_query("select id from domains where title = '$domain'", $mysql_connect);
 
        if (!mysql_num_rows($res)) {
            mysql_query("insert into domains (title) values ('$domain')", $mysql_connect);
 
            mail('your.email@yourdomain.com', "Domain Collector Notification", 'New domain: ' . $domain);
        }
 
        mysql_close($mysql_connect);
    }
}
?>

Linkpot Plugin

The plugin hits Domain Collector Web Service (POST) when someone posts a comment and the comment’s approval status is not ’spam’.

Find the plugin here. It should work with the latest Wordpress version. Upload, install and specify params.

I didn’t add the WP Mu support. If you need that, just do it.

All Together

Trolls don’t know domains and links are being collected. One day you’ll spam their blogs and blogs they spam.

PS: Of course, this is not the only way to populate domains & links database. You can find/buy/scrape similar databases and import data. *tip spam tool tip*

PPS: fromzerotoseo.com is powered by Linkpot 

Implementation tips

1. Create similar structure.

2. Add some domains (processed = 0).

1|johnchow.com|0
2|yourblogname.com|0
…
N|weightlossspamblog.com|0

3. Get domains’ links; extract domains from links, (filter), save domains and links.

$domains = select id, title from domains where processed = 0;
foreach ($domains as $d) {
    $links = get links;
    foreach ($links as $link) {
        $domain = extract from $link['url'];
        if ($domain in [.com .net .org …] && not in [.cn …] && no [viagra cialis …]) {
            insert into domains (title, processed) values ($domain, 0);
            insert into links (url, title, domain_id) values ($link['url'], $link['title'] $d['id']);
        }
    }
    update domains set processed = 1 where id = $d['id'];
}

4. Loop/cron 3 without hitting the limit.

What next?

Time and multiple IPs are your friends. After a week or two do something like

$links = select url, title from links where url like '%keyword%'

or

$links = select url, title from links where title like '%keyword%'

or

$links = select url, title from links where title like '%keyword%'
         and url like '%keyword%'

You got the idea. Then

foreach ($links as $link) {
    if ($link from wordpress) {
        spam $link;
    }
}

or

foreach ($links as $link) {
    if ($link from wordpress && pingback allowed) {
        PingCrawl $link;
    }
}

Next time some tips on collecting domains.

Find the spammer here.

Code example (http://www.bbpressthemes.net/navigation/)

Same shit in Lynx

P.S.: Google Webmaster Tools > Labs > Fetch as Googlebot reveals shit as well.

Bullshit you say? OK then what about a 25 days old whitehat (unique contents & design) blog ranking #20+ for its main keyword out of 8.6 millions? Again, few bookmarks, 80+ pages in index, and, the most important factor, main keyword in domain name.

Where must be some kind of trick? You right. You can make your beast get ranks faster. Won’t tell you all the details, but YouTube, Wikipedia, sub domains, trusted competitors’ websites and daily posting will help it. BUT, after you get some good domains.

BTW, I would move 3 -> 1.

P.S.: Another autogenny baby keeps going up. Same techniques.

I won’t post make-money-bullshit about the scraper. Watch video and make a decision – to buy or not to buy Price is high, but the program worth moniz. Also try to find discount codes. I know two places: wickedfire and warriorforum.

Cheers!

Solution Overview

You need a pair of scrapers. One of them uses search functionality. It grabs search results, URLs and titles, and saves them to database. Another scraper takes URLs (and titles) from the database and scraps articles.

Why two scrapers instead of one? Because running multiple instances of both scrapers, i.e. scraping different websites in parallel, reduces the risk of getting caught, but, at the same time, allows you to pull out many articles. Directories are happy, because you’re whoring them politely and you’re happy, because you get many articles fast.

Two scrapers per hundreds of directories might be a challenging coding task. I recommend you to write bots so they can get websites related data from the database. Don’t use files and command line arguments too much, because it’s a pain in the ass. Create the design constraint – ‘100% code reuse’ and never violate it. You will reuse this bitch many times over and over again without changing single line of code. It pays off, because when you want to scrap another directory, you simply add one database record and run the scripts. Got it? OK, it’s time to get your hands dirty.

Searching Directories

Don’t type just “article website” or “article directory”. Target a group of websites or platforms, because you need a single solution for multiple websites.

• “Total Articles:” AND “Total Authors:”
• There are * “published articles and” * “registered authors in our article directory.”
• “Powered by WordPress • Using Article Directory plugin”
• “Powered By: Article Friendly”

Select two or three websites. It’s enough for you to start with. Later you will screw them all.

Creating Database

The structure is pretty self-evident, but some fields will be explained later.

articles_websites table:

1. Website Id – auto increment primary key;
2. Website name – domain is OK;
3. Search URL;
4. Next Page regex;
5. Titles URLs regex;
6. Article regex.

search_results table:

1. Article URL – unique index;
2. Article Title;
3. Website name or website Id – foreign key articles_websites.[Website Id].

articles table:

1. Article Id – auto increment primary key;
2. Article Title;
3. Article text;
4. Website name or website Id – foreign key articles_websites.[Website Id].

Whoring Search

Take each selected directory and fill in the articles_websites table:

1. Find out how search URL looks like. Should be something like http://www.website.com/search=keywords. Save it to articles_websites.[Search URL] without ‘keywords’ or replace ‘keywords’ with the %s (depends on the way you build URL).
2. Write regular expression, which extracts “Next page” URL from the search results page. Usually search results are paginated and there is the button Next/NEXT/>/etc. Play with search and keywords if you don’t see the button. Save regex to articles_websites.[Next Page regex].
3. Write regular expression, which extracts all Title+URL pairs (search results) from the search results page. Usually this regex is trickier than previous, but it’s 100% feasible. Save it to articles_websites.[Titles URLs regex].
4. Open any article and write regular expression, which extracts article content. Save it to articles_websites.[Article regex].

Steps 1-4 is everything I need to do to kill another victim. Guess, how much directories you can add in one hour? “Suck boobs and let scrapers suck content” principle in action

Coding Search Results Scraper

Input: website name and keyword(s).

1. Load website’s data from database.
2. For each keyword(s) build URL replacing %s and scrap the search results page.
3. Extract Next Page URL.
4. Extract all Titles+URLs; save them to the search_results table.
5. Timeout.
6. While Next Page URL not empty repeat 2-6.

Output: Titles+URLs saved in the search_results table.

Coding Articles Scraper

Input: website name.

1. Get Titles+URLs from the search_results table where website name is [website name].
2. For each Title+URL scrap URL.
3. If success, then extract article (Article regex), save Title+Article+Website Name (Or Id) to the articles table. Delete record URL+Title from the search_results table.
4. Timeout.

Output: Articles saved in the articles table.

Implementation Tips

1. Rotate/select random proxies while scraping.
2. Always timeout and timeout wisely. The point is to run several instances of the scrapers and scrap different websites in parallel, but not to scrap one website every second.
3. Strip of scripts and tags from article before saving/posting.
4. Find the optimal scraping rate and schedule scrapping.

Happy coding!

Get Google SERP and preg match local business results with

$page = file_get_contents('http://www.google.com/search?q=new+york+pizza');
 
preg_match_all(
    '@<h4[^>]*class=r[^>]*>\s*<a\s*href="(.*)".*class=l.*title="(.*)".*>@siU',
    $page,
    $matches);
 
print_r($matches[1]); // urls
print_r($matches[2]); // titles

Then preg match regular listings (I hope regex works) and go through both $matches, local and regular.

HTH

Scrapers are common. Grab a website or page, extract data, save it. See logical parts? Good. You got the idea.

Building SERP proscraper

Pro: How many search engines do you know?
Noob: About 20.
Pro: How many of them do you scrap?
Noob: Five. Google, Yahoo, Bing, Ask and Cuil.
Pro: And how many scrapers do you have?
Noob: Five. [It's pain in the ass to build them.]
Pro: Five?! I have only one server side scraper for all engines. And I can easily extend the scraper in no time.
Noob: Teach me! Do you have a ebook? [I'll be your pet. I'll suck your balls… Why not?]
Pro: [F*ck off!] Learn design patterns and quit reading gay forums.
Noob: I’ll do what you say. I want to be a blackhat.
Pro: [Hate noobs…] Read carefully. You’ll be a SERP scraper hero soon.

A closer look at SERP scrapers

Haven’t open the Noob’s links yet? Do it now. Can you identify the common parts and the parts that vary? The getPage part is common, but the extraction parts vary. E.g. it’s one-to-one relationship between search engine and parsing-extraction part; and one-to-many relationship between the getPage and search engines. What does it mean? It means – separate the getPage part and the extraction part.

Coding SERP Parser Factory

Factory is the best choice for coding SERP parsers by all means. You are passing the type of search engine to the factory and based on that type the SERP parser factory creates the concrete SERP parser. When you need new search engine scraper, you create concrete SERP parser class and add two lines to the SERP parser factory. Checkmate!

Here are some code snippets for you. Now you have a head start on noobs.

iSerpParser interface (i_serp_parser.php)

interface iSerpParser {
    public function parse($serp);
}

Google SERP parser – example of concrete SERP parser (google_serp_parser.php)

include_once('i_serp_parser.php');
 
class GoogleSerpParser implements iSerpParser {
 
    public function parse($serp) {
       // necessary code …
       return $serpData;
    }
 
    // other methods …
}

Yahoo SERP parser – example of concrete SERP parser (yahoo_serp_parser.php)

include_once('i_serp_parser.php');
 
class YahooSerpParser implements iSerpParser {
 
    public function parse($serp) {
       // necessary code …
       return $serpData;
    }
 
    // other methods …
}

And finally, the SerpParserFactory (serp_parser_factory.php)

include_once('google_serp_parser.php');
include_once('yahoo_serp_parser.php');
// include concrete SERP parsers …
 
class SerpParserFactory {
    public static function createSerpParser($searchEngineName) {
        $serpParser = NULL;
 
        if ($searchEngineName == 'google') {
            $serpParser = new GoogleSerpParser();
        } elseif ($searchEngineName == 'yahoo') {
            $serpParser = new YahooSerpParser();
        }
        // create concrete SERP parser and assign to $serpParser …
        return $serpParser;
    }
}

The SERP proscraper coding is almost done. Find out which engine, get page, parse and save (serp_scraper.php)

include_once('serp_parser_factory.php');
 
// get engine name from the command line, database …
$seName = getSearchEngine();
 
// getPage…
$page = getPage(…);
 
// create SERP parser
$serpParser = SerpParserFactory::createSerpParser($seName);
 
// parse SERP
$data = $serpParser->parse($page);
 
// save data …

Aftercoding chat

Pro: Now you’re prepared for SERP scraping battle.
Noob: I can add as many engines as I want. F*king easy! I have to code only SERP parsing class. Thanks Pro! Could you share more blackhat tips? [Teach me and I …]
Pro: Dude, it’s just the beginning. You can scrap thousands articles if you switch your stupid WH/BH mindset to coding mindset. But don’t go out to sea before you’ve learned how to swim. Study coding and design patterns: Five common PHP design patterns, Recommended PHP reading list, Head First Design Patterns (aff).
Noob: Thanks man!

I’m looking back at myself, at that idiot, who thought that making cash online is easy. Huge plans… :} Now I see where I’m and the way I have to go to achieve my goal. It takes longer than I thought. But we all start from Zero. And there is nothing wrong with it.

Cheers!