cosmicsea
03-25-2010, 04:56 PM
I have a crawler which crawls for links and i want to make it login to sites that require you to login to view them. my crawler has this function in the phpcrawler.class.php file
// Method adds an authentication-login for special URLs given in expression
// (PCRE)
function addBasicAuthentication($expression, $username, $password)
{
$this->initCrawler();
$check = PHPCrawlerUtils::checkExpressionPattern($expression); // Check pattern
if ($check == true)
{
$c = count($this->pageRequest->basic_authentications);
$this->pageRequest->basic_authentications[$c]["match"] = $expression;
$this->pageRequest->basic_authentications[$c]["username"] = $username;
$this->pageRequest->basic_authentications[$c]["password"] = $password;
return true;
}
else return false;
}
and here is checkExpressionPattern from PHPCrawlerUtils.class.php
// Function checks if a given preg_pattern is a valid one.
function checkExpressionPattern($pattern)
{
$check = @preg_match($pattern, "anything"); // thats the easy way to check a pattern ;)
if (is_integer($check) == false) return false;
else return true;
}
}
with this info is it possible to create something like that? if so can somebody help me? i would appreciate it. Thanks.
// Method adds an authentication-login for special URLs given in expression
// (PCRE)
function addBasicAuthentication($expression, $username, $password)
{
$this->initCrawler();
$check = PHPCrawlerUtils::checkExpressionPattern($expression); // Check pattern
if ($check == true)
{
$c = count($this->pageRequest->basic_authentications);
$this->pageRequest->basic_authentications[$c]["match"] = $expression;
$this->pageRequest->basic_authentications[$c]["username"] = $username;
$this->pageRequest->basic_authentications[$c]["password"] = $password;
return true;
}
else return false;
}
and here is checkExpressionPattern from PHPCrawlerUtils.class.php
// Function checks if a given preg_pattern is a valid one.
function checkExpressionPattern($pattern)
{
$check = @preg_match($pattern, "anything"); // thats the easy way to check a pattern ;)
if (is_integer($check) == false) return false;
else return true;
}
}
with this info is it possible to create something like that? if so can somebody help me? i would appreciate it. Thanks.