PHP cURL proxy checker

Do you still read my blog? Good 🙂 This time I’m going to post some juicy code that can hide your ass.

Anonymity must be protected, don’t you agree? I’m not paranoid, but I use a bunch of anonymous proxies for server-side tasks. And I check them regularly.

Most of the time I use server-side PHP scripts, because desktop proxy checkers don’t provide much flexibility. You don’t want to check, export and upload a list of proxies every time you need them, do you? That’s why PHP cURL is the best choice for me. Automation is the king!

So, how to check proxies with PHP? Let’s start with the proxy basics.

What is an anonymous proxy?

An anonymous proxy hides your IP address. Sometimes it informs a client that a proxy is being used. Sometimes it looks like a real host/user.

What should I look for when checking proxy anonymity?

Make sure the real IP address is hidden. Check all variables that can expose your identity.

Warning! I’m not a security expert. Please let me know if any variable is missing from the list.

  • REMOTE_ADDR
  • REMOTE_HOST
  • HTTP_X_FORWARDED_FOR
  • HTTP_VIA
  • HTTP_CLIENT_IP
  • HTTP_PROXY_CONNECTION
  • FORWARDED_FOR
  • X_FORWARDED_FOR
  • X_HTTP_FORWARDED_FOR
  • HTTP_FORWARDED
  • HTTP_REFERER

Printing “bad” variables

Here is the first script (badvars.php) you need. It prints out required variables. Upload the file to a server and check if it’s working.

<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=UTF-8">
        <title>I know you</title>
    </head>
    <body>
        <p><?php echo("REMOTE_ADDR " . $_SERVER['REMOTE_ADDR'] . "\n"); ?></p>
	<p><?php echo("REMOTE_HOST " . $_SERVER['REMOTE_HOST'] . "\n"); ?></p>
	<p><?php echo("HTTP_X_FORWARDED_FOR " . $_SERVER['HTTP_X_FORWARDED_FOR'] . "\n"); ?></p>
	<p><?php echo("HTTP_VIA " . $_SERVER['HTTP_VIA'] . "\n"); ?></p>
	<p><?php echo("HTTP_CLIENT_IP " . $_SERVER['HTTP_CLIENT_IP'] . "\n"); ?></p>
	<p><?php echo("HTTP_PROXY_CONNECTION " . $_SERVER['HTTP_PROXY_CONNECTION'] . "\n"); ?></p>
	<p><?php echo("FORWARDED_FOR " . $_SERVER['FORWARDED_FOR'] . "\n"); ?></p>
	<p><?php echo("X_FORWARDED_FOR " . $_SERVER['X_FORWARDED_FOR'] . "\n"); ?></p>
	<p><?php echo("X_HTTP_FORWARDED_FOR " . $_SERVER['X_HTTP_FORWARDED_FOR'] . "\n"); ?></p>
	<p><?php echo("HTTP_FORWARDED " . $_SERVER['HTTP_FORWARDED'] . "\n"); ?></p>
	<br>
	<p><?php echo("HTTP_REFERER " . $_SERVER['HTTP_REFERER'] . "\n"); ?></p>
	<p><?php echo("HTTP_USER_AGENT " . $_SERVER['HTTP_USER_AGENT'] . "\n"); ?></p>
    </body>
</html>

FYI, I’ve also included HTTP_USER_AGENT.

What is my IP?

You need to know your IP address before checking anonymity. If you have only one server it’s not a problem. Hardcode and forget. But what if you have several servers? As I said before, automation is the king.

Say thank you to people who cares about us. Grab URL(s) you like, extract IP and be happy. The WhatIsMyIp class (whatismyip.php) will help you in the beginning.

<?php
class WhatIsMyIp {
    private function extractIpAddress($text) {
        $ip = NULL;
 
        if (preg_match(
                "/(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)\.(25[0-5]|2[0-4][0-9]|[01]?[0-9][0-9]?)/",
                $text, $matches)) {
            $ip = $matches[0];
        }
        return $ip;
    }
 
    public function getServerIpAddress() {
        $file = file_get_contents(/*[URL YOU LIKE]*/);
        return $this->extractIpAddress($file);
    }
}
?>

Checking proxy

Finally, the last part of the post. Get ready for some “serious” programming 🙂

Copy proxychecker.php.

<?php
/*
 * TODO 1
 * Include whatismyip.php. Don’t forget to specify URL.
 */
 
/*
 * TODO 2
 * Copy getPage (http://www.fromzerotoseo.com/scraping-websites-php-curl-proxy/)
 * here or include it as a file. 
 */
 
set_time_limit(/* TODO 3 specify a time limit */);
 
function getProxies() {
    /*
     * TODO 4
     * Load proxies from a file or database.
     * Return array of [IP]:[port] proxies or NULL. 
     */
}
 
function deleteProxy($proxy) {
    /*
     * TODO 5
     * Delete $proxy from file/database or
     * mark as 'bad'.
     */
}
 
$whatIsMyIp = new WhatIsMyIp();
$myIpAddress = $whatIsMyIp->getServerIpAddress();
if (empty($myIpAddress)) {
    echo("Can't obtain IP address\n");
    exit();
}
 
$proxies = getProxies();
if (empty($proxies)) {
    echo("Can't load proxies\n");
    exit();
}
 
foreach ($proxies as $proxy) {
    $result = getPage(
        $proxy,
	/*
	 * TODO 6
	 * Specify badvars.php URL 
	 */,
	'http://www.google.com/',
        'Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.8) Gecko/2009032609 Firefox/3.0.8',
	1,
	5);
 
    if (empty($result['ERR'])) {
        if (preg_match("/" . $myIpAddress . "/", $result['EXE'])) {
            deleteProxy($proxy);
        } else {
            /*
             * Good proxy!
             */
        }
    } else {		
        echo($result['ERR'] . '\n');
	deleteProxy($proxy);
    }
 
    sleep(rand(/* TODO 7 [int] */, /* [int] */));
}
?>
  1. Include whatismyip.php. Don’t forget to specify URL.
  2. Copy getPage function to proxychecker.php or include it as a file.
  3. Specify a time limit.
  4. Write getProxies() function.
  5. Write deleteProxy($proxy) function.
  6. Specify badvars.php URL.
  7. Specify sleep intervals.
  8. Test and fix proxychecker.php. You don’t need to upload the script to a server. You can use your PC for testing.
  9. Deploy the script. Don’t use the same host for badvars.php and proxychecker.php.

See ya!

12 thoughts on “PHP cURL proxy checker”

  1. these scraping tutorials have been excellent

    the only thing i am not familiar with is proxies… im not sure if i am looking at this all wrong, i understand the importance of proxies, but am not 100% sure what this script is doing is it grabbing ip address from whatismyip to use as fake ips to use as proxies?

    sorry not had my morning coffee yet… so everything looks a little hazy right now

  2. Not really. WhatIsMyIp returns computer’s IP address, there the proxychecker.php is. ipconfig does the same job. Then proxychecker.php gets (using curl and proxy) the badvars.php page and looks for computer’s IP address. If no IP then OK.

    The problem is that I use different computers and networks for scrapping and etc.. For example, public wifi and laptop :] Each time I check proxy anonymity.

    I don’t want to use ipconfig and setup IP manually ->

    preg_match(“/” . $myIpAddress . “/”, $result[‘EXE’])

    That’s my WhatIsMyIp is exactly what I need.

  3. I’ve started the code, and added “http://www.google.com” in the what is myip function, but i get the error, can’t obtain ip address – i’ve tried to put in my own websites, but i get the same result.

    Any ideas what i’m doing wrong?

  4. Just wanted to extend my thanks for this, you just saved me a couple of hours work laying out the framework to check proxys from a DB

  5. Okay dude, I am seriously impressed. Jealous in fact of your prolific programming prowess. You are a god amongst ants.

    Now, do you want to really, really, really impress me? I know you do. So here’s how in two simple, easy peasy weasy wittle step:

    1. Modify todo 4:

    /*
    * TODO 4
    * Load proxies from a file or database.
    * Return array of [IP]:[port] proxies or NULL.
    */

    To this:

    /*
    * TODO 4
    * Load proxies by scraping various listings from the Net via google search like so: http://www.google.com/search?hl=en&safe=off&client=safari&rls=en&q=proxy+list+free+port+high+anonymity+new+zealand+%2B%22united+states%22+china&aq=f&oq=&aqi=
    * Process all the proxies to see what’s good.
    * Store those in a db to be tested at interval
    * Return array of [IP]:[port] proxies or NULL.
    * Rinse and repeat
    */

    2. Share code with seozero admirers, or alternatively create an API that admirers can query against for the spiffiest, freshyist proxies ever!

    …Or, recommend some other way to automate grabbing good proxies (even paid).

    Me wants!

    :))

  6. you don’t need a whatismyip function just upload this script to a server other than the one with the proxychecker.php script

    your own ip checker web page 🙂

    that’ means you don’t have to extract anything, just get your ip like this:

    $myIpAddress = get_file_contents( $ipurl );

  7. didn’t print the script in my last comment, this is it:

    print $_SERVER[‘REMOTE_ADDR’];

    in php tags of course

  8. hey, really cool blog you have.

    any tips you have on a way i can make this multithreaded?
    it works but it i have a huge list

Leave a Reply

Your email address will not be published. Required fields are marked *