Skip to content

Commit

Permalink
Initial add.
Browse files Browse the repository at this point in the history
  • Loading branch information
@icculus
icculus committed Nov 7, 2015
0 parents commit 4b89a01
Show file tree
Hide file tree
Showing 8 changed files with 1,479 additions and 0 deletions.
3 changes: 3 additions & 0 deletions .hgignore
@@ -0,0 +1,3 @@
syntax:glob
config.php
followers
23 changes: 23 additions & 0 deletions LICENSE.txt
@@ -0,0 +1,23 @@

Copyright (c) 2012 Ryan C. Gordon.

This software is provided 'as-is', without any express or implied warranty.
In no event will the authors be held liable for any damages arising from
the use of this software.

Permission is granted to anyone to use this software for any purpose,
including commercial applications, and to alter it and redistribute it
freely, subject to the following restrictions:

1. The origin of this software must not be misrepresented; you must not
claim that you wrote the original software. If you use this software in a
product, an acknowledgment in the product documentation would be
appreciated but is not required.

2. Altered source versions must be plainly marked as such, and must not be
misrepresented as being the original software.

3. This notice may not be removed or altered from any source distribution.

Ryan C. Gordon <icculus@icculus.org>

51 changes: 51 additions & 0 deletions README.txt
@@ -0,0 +1,51 @@
To set this up:

(These dev.twitter.com instructions are right as of November 2012, but the UI
could always change. Wing it.)

- Go to dev.twitter.com, login, go to "My applications" on the menu that pops
up when you mouse over your avatar on the top right of the page.
- Click the "Create a new application" button.
- Fill in whatever you want here. You can leave "Callback URL" blank. Agree to
the license and click the create button.
- When you app is created, click "Create my access token" at the bottom of the
next page.

This app only needs to be read-only; it never tries to post or edit anything
with your account. We just need this so we can grab your follower list from
Twitter.

Now you should have four magic values on that page: Consumer Key,
Consumer Secret, Access Token, and Access Token Secret.

Make a file in this directory called config.php, with these four magic values,
and your screen name, like this:


<?php
define('CONSUMER_KEY', '3k9sjcS4thSfsfsgW');
define('CONSUMER_SECRET', 'AKCdfsdfSF9sdf98989sdfFSsFQasdfokfAdsfqFRTAcVx');
define("OAUTH_TOKEN", '34672782-wwgovSDFSdf9sdf0DFSFLf08afamkg2GZga';
define("OAUTH_SECRET", 'sf0fkv9s82sffah2k3333FSFsfkskaf9aFAf9faghjQ');
define("TWITTER_USERNAME", 'icculus');


Now run "php ./twitterlosses.php" and if all went well, it should pull in all
of your current users, assuming you aren't like Beyonce famous. This first
run or so will dump out a lot of information, and cache the users it sees.
Once you are set up, future runs will just report changes from what is already
known (that is, new people following and accounts that unfollow you).

You'll want to run twitterlosses.php in a cronjob, so new info shows up
whenever the script sees it. I use the included mailupdate.sh script in a
cronjob every 15 minutes, but you can do whatever you like.

You may need to sniff around for hardcoded things. Grep for 'icculus' to
fix a hardcoded URL or two. Send patches.

Questions: ask me.

--ryan.

icculus@icculus.org

22 changes: 22 additions & 0 deletions TWITTEROAUTH-LICENSE.txt
@@ -0,0 +1,22 @@
Copyright (c) 2009 Abraham Williams - http://abrah.am - abraham@poseurte.ch

Permission is hereby granted, free of charge, to any person
obtaining a copy of this software and associated documentation
files (the "Software"), to deal in the Software without
restriction, including without limitation the rights to use,
copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the
Software is furnished to do so, subject to the following
conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
OTHER DEALINGS IN THE SOFTWARE.
12 changes: 12 additions & 0 deletions mailupdate.sh
@@ -0,0 +1,12 @@
#!/bin/bash

X=`which "$0"`
Y=`readlink -e "$X"`
cd `dirname "$Y"`
/usr/bin/php ./twitterlosses.php > dump.txt
if [ -s 'dump.txt' ] ; then
( echo -e 'From: TwitterLosses <icculus@icculus.org>\nTo: icculus@icculus.org\nSubject: Twitter follower update (icculus)\nContent-Type: text/plain; charset=UTF-8\n\n' ; cat dump.txt ) |/var/qmail/bin/qmail-inject
fi

rm -f dump.txt

249 changes: 249 additions & 0 deletions twitterlosses.php
@@ -0,0 +1,249 @@
<?php
/* Load required lib files. */
require_once('twitteroauth/twitteroauth.php');
require_once('config.php');


function cache_users(&$userobjs, $cachedir, $idsstr)
{
global $connection;

//print("Getting $idsstr ...\n");

$data = $connection->get('users/lookup', array('user_id' => $idsstr, 'include_entities' => 'true'));
if ($data === false)
{
print("get users failed:\n");
print(" - HTTP connection totally failed. Network burp?\n");
return;
}
else if (isset($data->errors))
{
print("get users failed:\n");
foreach ($data->errors as $err)
print(" - {$err->message}\n");
return;
}

//print_r($data);
foreach ($data as $obj)
{
unset($data->status); // don't care about this.
$fname = $obj->id_str;
$username = $obj->screen_name;
$cachefname = "$cachedir/$fname";
$obj->account_was_deleted = false;
if (file_put_contents($cachefname, serialize($obj)) === false)
{
print("couldn't write $cachefname (user $username)!\n");
if (file_exists($cachefname))
unlink($cachefname);
continue;
}

$userobjs[] = $obj;
}
} // cache_users

function print_user_object($obj)
{
if ($obj->account_was_deleted)
print("*** THIS ACCOUNT APPEARS TO HAVE BEEN DELETED. ***\n"); // POSSIBLY A SPAMMER.
print("real name: {$obj->name}\n");
print("screen name: {$obj->screen_name}\n");
print("description: {$obj->description}\n");
print("location: {$obj->location}\n");
print("url: {$obj->url}\n");
print("twitter url: https://twitter.com/{$obj->screen_name}\n");
print("twitter url-by-id: https://twitter.com/intent/user?user_id={$obj->id_str}\n");
print("tweet count: {$obj->statuses_count}\n");
print("id: {$obj->id_str}\n");
print("lang: {$obj->lang}\n");
print("created: {$obj->created_at}\n");
print("following: {$obj->friends_count}\n");
print("followers: {$obj->followers_count}\n");
print("favorites: {$obj->favourites_count}\n");
print("timezone: {$obj->time_zone}\n");
print("are following: " . (($obj->following == '1') ? 'yes' : 'no') . "\n");
print("protected: " . (($obj->protected == '1') ? 'yes' : 'no') . "\n");
print("verified: " . (($obj->verified == '1') ? 'yes' : 'no') . "\n");
print("\n");
} // print_user_object


function print_user_objects($type, &$userobjs)
{
if (count($userobjs) == 0)
return;

print("*** $type:\n\n");
foreach ($userobjs as $obj)
print_user_object($obj);
} // print_user_objects


function print_cached_user_objects($type, $cachedir, $ids)
{
$userobjs = array();
foreach ($ids as $id)
{
$cachefname = "$cachedir/$id";
$data = file_get_contents($cachefname);
if ($data === false)
print("couldn't read $cachefname!\n");
else
{
$obj = unserialize($data);
if ($obj === false)
print("couldn't unserialize $cachefname!\n");
else
$userobjs[] = $obj;
}
}

print_user_objects($type, $userobjs);
unset($userobjs);
} // print_cached_user_objects

function check_if_users_still_exist($cachedir, $ids)
{
foreach ($ids as $id)
{
$cachefname = "$cachedir/$id";
$data = file_get_contents($cachefname);
if ($data === false)
{
print("couldn't read $cachefname!\n");
continue;
}
$obj = unserialize($data);
if ($obj === false)
{
print("couldn't unserialize $cachefname!\n");
continue;
}

// this is not foolproof, of course, but it's good enough.
//print("Checking if {$obj->screen_name} ({$obj->name}) still exists...");
$handle = popen("curl -I 'https://twitter.com/intent/user?user_id={$obj->id_str}' 2>/dev/null |head -n 1 |perl -w -p -e 's/\AHTTP\/1.1 (\d+) .*?\Z/$1/;'", 'r');
$rc = intval(fgets($handle));
pclose($handle);
//print("$rc\n");
if ($rc == 404)
{
$obj->account_was_deleted = true;
file_put_contents($cachefname, serialize($obj));
}
}
} // check_if_users_still_exist


/* Create a TwitterOauth object with consumer/user tokens. */
$connection = new TwitterOAuth(CONSUMER_KEY, CONSUMER_SECRET, OAUTH_TOKEN, OAUTH_SECRET);

/* If method is set change API call made. Test is called by default. */
$content = $connection->get('account/verify_credentials');

$twitterids = array();

$cursor = '-1';
while ($cursor != '0')
{
//print("cursor == $cursor\n");
$data = $connection->get('followers/ids', array('screen_name' => TWITTER_USERNAME, 'count' => '5000', 'cursor' => "$cursor", 'stringify_ids' => 'true'));
if ($data === false)
{
print("get followers (cursor=$cursor) failed:\n");
print(" - HTTP connection totally failed. Network burp?\n");
exit(1);
}
else if (isset($data->errors))
{
print("get followers (cursor=$cursor) failed:\n");
foreach ($data->errors as $err)
print(" - {$err->message}\n");
exit(1);
}

foreach ($data->ids as $id)
$twitterids[$id] = $id;

//print_r($data);

$cursor = $data->next_cursor_str;
}

//print("saw " . count($twitterids) . " ids\n");

$cachedir = 'followers';
if (!file_exists($cachedir))
mkdir($cachedir);

$dirids = scandir($cachedir);
if ($dirids !== false)
{
while (($dirids[0] == '.') || ($dirids[0] == '..'))
array_shift($dirids);
}

$cachedids = array();
foreach ($dirids as $id)
$cachedids[$id] = $id;
unset($dirids);

$newfollows = array();
foreach ($twitterids as $id)
{
if (!isset($cachedids[$id]))
$newfollows[] = $id;
}

$unfollows = array();
foreach ($cachedids as $id)
{
if (!isset($twitterids[$id]))
$unfollows[] = $id;
}

//print_cached_user_objects("All followers", $cachedir, $cachedids);
unset($twitterids);
unset($cachedids);

$userobjs = array();
$count = 0;
$idsstr = '';
foreach ($newfollows as $id)
{
if ($id == '109134780') // !!! FIXME: what is this?
continue;

if ($idsstr != '')
$idsstr .= ',';
$idsstr .= $id;
$count++;
if ($count >= 100)
{
cache_users($userobjs, $cachedir, $idsstr);
$idsstr = '';
$count = 0;
}
}

if ($count > 0)
cache_users($userobjs, $cachedir, $idsstr);

check_if_users_still_exist($cachedir, $unfollows);
print_cached_user_objects("Unfollows", $cachedir, $unfollows);
print_user_objects("New follows", $userobjs);

foreach ($unfollows as $id)
{
$cachefname = "$cachedir/$id";
if (file_exists($cachefname))
unlink($cachefname);
}

exit(0);

?>

0 comments on commit 4b89a01

Please sign in to comment.