Twitter alerts: using twitter streaming API for fun and profit
23Oct 2009
Twitter is a wonderful service, but, until now, you have to subscribe to some websites to be alerted when a selected word (maybe your trademark) is tweeted. We’ll try to develop a service that filters the tweeter api, stores the interesting ones in our database, and show them in the browser in real time.
NOTE: Now you have to use https:// instead of http:// for it to work.
What?
We’ll take twitter real time results for a given word (or words) and visualize them in a browser window, like monitter.com, but on our own servers and a bit more automatic. This is going to be useful to get our own alerts and work with them.
How?
We are going to use the twitter stream API to get the comments containing a given word. Since the results are given in Json format, we’ll need to filter the data, take the interesting bits and store them. From the other side, we’re going to write a bit of javascript to read the data from the server into a web page using ajax. We could use other technologies, like comet, to push the data to the browser, but, while it would be a much cleaner implementation, I think I’m not ready to write about that yet (check this space ). We need to decouple the reading of the stream from twitter and the serving to our clients because we can only have a single active instance of the twitter streaming api at a given time, and we’re going to leave the listening proccess on for a long time using a daemon/service approach. On top of that, we want to store the tweets in a database for later perusal and data mining.
The tools
We are trying to make a useful system. To do this, we’ll need some tools:
HTTP server:I’m currently using Apache, but any one would be OK
Server side programming language: This time, PHP. While probably not as elegant or fast as Python or as cool as Ruby, gets the work done and it’s available in almost all web hosting plans. And the documentation is pretty extensive.
A database to store all the info
Client side programming: We are going to use HTML, javascript and the jQuery javascript library to simplify AJA(X) programming and for the effects.
A twitter acount: Yours own is OK but maybe you want to create a new one for this kind of tasks. Write down your user and password, we are going to need them soon.
The twitter API. Twitter people are so kind they have developed a restful API free for everyone to use. And it’s sub-zero cool.
The Twitter streaming API
Twitter has published a Streaming API that’s described as “The Twitter Streaming API allows near-realtime access to various subsets of Twitter public statuses”. In fact, this is jus what we need. You can read all the documentation at https://twitterapi.pbworks.com/Streaming-API-Documentation, but I’ll try to take the interesting parts for this project so you don’t need to yet.
We are going to use just one method (status/filter) to get results including one or several words. This method can return a stream of data in xml or json formats, has to be called using POST and can get some parameters. You can use it from the command line if you have access to some kind of unix in the following way:
{"text":"RT @gillesguillemin: Google FINALLY Releases AS3 Player for YouTube http://bit.ly/yKgaz",
"favorited":false,"in_reply_to_user_id":null,"in_reply_to_screen_name":null,"source":"web","geo":null,
"in_reply_to_status_id":null,"user":{"friends_count":52,"screen_name":"imrahil","verified":false,
"profile_background_color":"9ae4e8","favourites_count":8,"notifications":null,"profile_text_color":"000000",
"description":"","location":"Poland","time_zone":"Warsaw","profile_link_color":"0000ff","following":null,
"profile_background_image_url":"http://s.twimg.com/a/1255558003/images/themes/theme1/bg.png",
"profile_sidebar_fill_color":"e0ff92","protected":false,"url":"http://flex.imrahil.com",
"geo_enabled":false,"profile_background_tile":false,"name":"Jarek","profile_sidebar_border_color":"87bc44",
"profile_image_url":"http://a3.twimg.com/profile_images/76229939/avatar5_normal.jpg","id":5393872,
"statuses_count":48,"utc_offset":3600,"created_at":"Sun Apr 22 07:02:54 +0000 2007","followers_count":31},
"id":4887052323,"truncated":false,"created_at":"Thu Oct 15 12:10:57 +0000 2009"}
{"text":"Google Wave\u2019s Little Secret: It Already Works On The iPhone","favorited":false,
"in_reply_to_user_id":null,"in_reply_to_screen_name":null,
"source":"<a href=\"http://www.atebits.com/\" rel=\"nofollow\">Tweetie</a>","geo":null,
"in_reply_to_status_id":null,"user":{"friends_count":52,"screen_name":"Jwizzman",
"verified":false,"profile_background_color":"000000","favourites_count":1,"notifications":null,
"profile_text_color":"663B12","description":"","location":"Amsterdam, The Netherlands",
"time_zone":"Amsterdam","profile_link_color":"1F98C7","following":null,
"profile_background_image_url":"http://a3.twimg.com/profile_background_images/20711299/background.png",
"profile_sidebar_fill_color":"DAECF4","protected":false,"url":"http://www.julianprofas.com",
"geo_enabled":false,"profile_background_tile":false,"name":"Julian Profas \uf8ff",
"profile_sidebar_border_color":"C6E2EE",
"profile_image_url":"http://a3.twimg.com/profile_images/458268969/foto_avatar_2_normal.png",
"id":15727906,"statuses_count":645,"utc_offset":3600,"created_at":"Mon Aug 04 21:08:08 +0000 2008",
"followers_count":79},"id":4887052409,"truncated":false,"created_at":"Thu Oct 15 12:10:58 +0000 2009"}
Until you exit it (with CTRL+C) . This is a Json stream and can be read and parsed by several means. In fact, it’s eval-uable javascript code that we could read from the browser. But right now, we’re going to use a server-side language to read it and work with the interesting parts.
Reading Twitter stream
As I told you in the tools section, we are going to use PHP as our server side language. For the first part, the reading of the twitter stream, we don’t even need a web server, since we can run it from the command line. And if we run it from the command line, we can convert it to a kind of daemon/service and leave it on for a long time. But first, some code:
This is (almost) the simplest code that delivers what we want. A php formatted stream of tweets that include our marked word. If you run it from your command line, it’ll show something like:
We create an $opts array (in fact an array of arrays) that contain the parameters. In this particular case, we’re using two, the POST method and the search line (track=google). Then, we can treat the twitter stream as a file, using stream_context_create and fopen and just start reading lines. Each line is going to be a JSON encoded tweet, similar to what we’ve seen when we called the API from command line. Since we want to use the contents as easily as possible, we’ll need the json_decode function to parse it into PHP objects, print them and call flush just in case we’re calling the script from a browser.
Storing the results
The best way to store the results for later perusal is a database. I’m using MySQL but any other database should be OK. To be able to store the data, we need to create a database with a single table.
We are only going to store the following data:
Text: This is the twetter status. 140 chars max.
User screen name: The screen name of the poster. This is needed to create the link to twitter
Id: A unique id for the tweet. It’s a sequential number, so, we can order the tweets acording to this, and use it along with the user screen name to built a link back to twitter, and use it as our primary key.
Followers count: The number of people that are going to receive the tweet in their inboxes. We are using it to style the real-time viewer. Since my primal intention is to watch a trademark, I care about the number of people that are watching the messages.
The time of the tweet: basically for filtering purposes. We are going to store our server time to avoid lengthy conversions.
We could store several other fields, and a complete solution should probably take into account that you can have some different tweet types, but for the time, these four fields should suffice.
To create the table, we can run the following SQL script from the server:
<?php
$opts = array(
'http'=>array(
'method' => "POST",
'content' => 'track=twitter',
)
);
//We're going to store the data in the database, so, let's open a connection:
$db = mysql_connect('localhost', 'twitter_alerts', 'somepasword');
mysql_select_db('twitter_alerts', $db);
$context = stream_context_create($opts);
while (1){
$instream = fopen('http://USERNAME:PASSWORD@stream.twitter.com/1/statuses/filter.json','r' ,false, $context);
while(! feof($instream)) {
if(! ($line = stream_get_line($instream, 20000, "\n"))) {
continue;
}else{
$tweet = json_decode($line);
//Clean the inputs before storing
$id = mysql_real_escape_string($tweet->{'id'});
$text = mysql_real_escape_string($tweet->{'text'});
$screen_name = mysql_real_escape_string($tweet->{'user'}->{'screen_name'});
$followers_count = mysql_real_escape_string($tweet->{'user'}->{'followers_count'});
//We store the new post in the database, to be able to read it later
$ok = mysql_query("INSERT INTO tweets (id ,text ,screen_name ,followers_count, created_at) VALUES ('$id', '$text', '$screen_name', '$followers_count', NOW())");
if (!$ok) {echo "Mysql Error: ".mysql_error();}
flush();
}
}
}
?>
It’s as ugly as sin but it works. If you run it from the command line, it should start storing tweets in your database and keep on until you stop it. So, our database is starting to fill with tweets concerning our desired word. Now we need to be able to navigate them…
Creating the code from the server side
Now we need to publish the tweets in our browser. To do that, we need a small PHP script that returns the tweets when called. If we call it with a parameter start it’ll return all the tweets with an id bigger than that. Otherwise, it’ll return the last ten tweets stored in our database. To do that, we will use two different queries, the first one to return the last ten results, and the second one to return all results since the given id. We use subqueries (SELECT from SELECT) to get the results in our wished order.
<?php
//We are going to need a database connection:
$db = mysql_connect('localhost', 'twitter_alerts', 'somepasword');
mysql_select_db('twitter_alerts', $db);
//Now, two possibilities: if we don't have a start parameter, we print the last ten tweets.
//Otherwise, we print all the tweets with IDs bigger than start, if any
$start = mysql_real_escape_string($_GET['start']);
if(! $start){
$query = "SELECT * FROM (SELECT * FROM tweets ORDER BY id DESC LIMIT 0,10) AS last_ten ORDER BY id ASC";
}else{
$query = "SELECT * FROM (SELECT * FROM tweets WHERE id>".$start." ORDER BY id DESC LIMIT 0,10) AS new_tweets ORDER BY id ASC";
}
$result = mysql_query($query);
$data = array(); //Initializing the results array
while ($row = mysql_fetch_assoc($result)){
array_push($data, $row);
}
$json = json_encode($data);
print $json;
?>
We are going to poll this code every ten seconds and refresh the tweets list to show the most recent ones, using javascript, and the output format will be JSON.
Writing a front-end
Since, as Larry Wall said, one of the cardinal virtues of a programmer is lazyness, we are going to use jQuery to construct the interface and the business logic. And we’re not even serving it, but linking from the Google CDN, as Dave Ward posted in his wondeful blog.
var last = '';
var timeOut;
function getTweets(id){
$.getJSON("server.php?start="+id,
function(data){
$.each(data, function(count,item){
addNew(item);
last = item.id;
});
});
}
function addNew(item){
if($('#tweets div.tweet').length>9){ //If we have more than nine tweets
$('#tweets div.tweet:first').toggle(300);//remove it form the screen
$('#tweets div.tweet:first').removeClass('tweet');//and it's class
$("#tweets div:hidden").remove(); //sweeps the already hidden elements
}
$('#tweets').append(renderTweet(item, 'hidden'));
}
function renderTweet(item){
importanceColor=getImportanceColor(item.followers_count);
return '<div class="tweet" id="'+item.id+'">'+
'<strong><a href="http://twitter.com/'+item.screen_name+'" style="color:'+importanceColor+'">'+
item.screen_name+'</a></strong><span class="text">'+
item.text
+'</span><span class="created_at"><br /><a href="http://twitter.com/'+
item.screen_name+'/status/'+item.id+'">'+
item.created_at+'</span></div>';
}
function getImportanceColor(number){
rgb = 255-Math.floor(16*(Math.log(number+1)+1)); //should return about 0 for 0 followers and 255 for 4million (Ashton Kutchner? Obama?)
return 'rgb('+rgb+',0,0)';
}
function poll(){
timeOut = setTimeout('poll()', 200);//It calls itself every 200ms
getTweets(last);
}
$(document).ready(function() {
poll();
});
Let’s see… This is probably the most complex part of the article, so, I’ll try to go slow and explain every function:
getTweets(id)
This function calls the server using the getJSON jQuery method. Then, it takes each response line and calls addNew with it. If we call it with an id parameter, it’ll ask the server for all tweets with ids greater than that. Otherwise, it will grab the last ten tweets.
addNew(item)
It takes an item (a tweet) as input. It hides the first tweet, remove it’s ‘tweet’ class, appends a new tweet at the bottom and shows it. It calls the renderTweet function to get the tweet in HTML format.
renderTweet(item)
Just one line to call getImportanceColor() and a return with the HTML code. It’s a bit long but that’s because we’re adding a couple of links to the tweet to be able to visit the original one.
getImportanceColor(number)
It takes a number of followers and returns a rgb color that will be between total black, for people without followers, and total red, for Ashton Kutchner. It uses logarithms to scale between the two extremes, because there are 6 orders of magnitude between the extremes. We will use it to paint (it) black the twitters with few followers and red the twitter stars.
poll()
This is the timeout function that calls itself every 200ms and gets the new tweets.
The last block just starts the polling as soon as the document is loaded.
The Result
This is a small screen capture of my browser visiting the HTML/javascript page while running storing_tweets_in_the_database.php. It’s watching the word ‘twitter’ and, as you can see, it’s running too fast for the human eye -at least mine -, but since we are keeping all the data in our database, it’s not lost forever
Limits
Right now, because of the Twitter API limits, just one instance of the watching process can be run at once. Anyhow, you can write several words, separated by commas, and Twitter will return results for all of them.This code should not be used in production, since there are almost no security checks to avoid missuse. If you want to use it in a machine open to the public, you should check -twice- every input for missbehaviour.
The code
Download the code and unzip it into a folder in your local webserver
Edit config.php to add your twitter login data and the words you want to watch
Create the database and the table with the SQL code above
Run watch.php and leave it running for as long as you wish.
Visit http://localhost/thefolderwhereyouunzippedthecode/ and watch the tweets coming.
Further work
Obviously, this is just a sample. It can be made much better looking, and we could even analyze the tweets and tweet back a response to any questions concerning our keywords. The watch module should be daemonized or converted to a service to be left unatended. The HTML page could be able to filter between two dates and so on. Keep on watching. We’ll try to keep on posting this kind of contents.
Shameless plug
I’m part of Corunet, a web agency in Spain, that can deliver consistent good results in all kind of internet projects. You can visit our website http://coru.net/ or contact me at david@corunet.com if you have any special needs
Thank you, plotti. Twitter has stated that they’re keeping the tweets, but they’re not available yet over the API. I’ll try to keep you informed if that changes.
This is the best tutorial on the Twitter Streaming API for PHP out there. Awesome job. I was a little disappointed to get to the end, have working code, and then find out that you don’t think this is ready for production. What do you think it would take to get his code ready for production? What are the issues? Have you looked at phirehose before?
I am trying to use the twitter streaming API for a personal project. If you think you could help me for a reasonable amount of compensation please contact me.
Hi Jed,
I don’t think it’s ready for production since it doesn’t take care of disconnections/reconnections, neither can update the stream for new filter words. Anyway, I’ve already used it with minor modifications for some customers. I’ve been trying phirehose lately and looks great, but can’t vouch for it yet.
If you want me to help you with a project, drop me a line to david@corunet.com with your idea and I will try to send you a budget.
Hi there, great article, i refer to your code a lot. Now im trying to figure out one thing, instead of the new tweet appears at the bottom and the top vanishes, how do i reverse that? mening to say newer tweets will show at top and the bottom one vanishes like tweetdec style.
I tried renaming first to last last to first but doesnt seem to work. Is it more complicated to start from top of issit very simple that i couldnt see it yet.
If u can kindly guide on how to reverse the tweets would be great. Thanks!
“Text: This is the twetter status. 140 chars max”
Thats just what the user is allowed to enter. If you take a look at the message, it contains of a lot more characters. That’s because links etc. are send as html.
I’m not at home so I cant give you the max number of what I have counted so far
Great tutorial! I have one question: in logic.js, when calling renderTweet(), you pass the function a second argument, ‘hidden’. I see what it achieves by inspecting the DOM in Firebug, but how does this take effect in your code? it’s a neat trick, but could you explain it? Thanks!
Hola david,
Me alegra ver que se hacen cosas majas en España también, he estado buscando información sobre el Streaming API de Google y es de los mejores tutoriales que he encontrado, mis felicitaciones.
Una pregunta, podemos hacer algo parecido sin tener que guardar datos en base de datos? es decir, mostrarlo a través de web según nos llegan de cURL?
Entiendo que en este caso no tendría sentido el php, no? Entiendo que el php no es algo que se quede “abierto” ejecutando algo, sería mas javascript o se te ocurre algo?
Muchas gracias y enhorabuena otra vez.
Fantastic, well written post about connecting to Twitter API. I have spent the afternoon playing around with this and I’ve learned a lot. No problems so far. Thank you.
Rico: as the dev.twitter.com URL says, you just have to change the ‘http’ by ‘https’.
If it ain’t working yet (as happened to me yesterday on this windows machine), then its because of configuration/setup.
Check that the: php_openssl extension is loaded and that you set allow_url_fopen = On in your php.ini
That should do it.
Good luck.
One question. What syntax should I use to combine predicates eg. add a track and a location? I’m trying things like… ‘track=’twitter&locations=360,180′ but not having success.
[...] Blog post that really helped me with the javascript part of the development: http://blog.corunet.com/twitter-alerts-using-twitter-streaming-api/ [...]
“You may export or extract non-programmatic, GUI-driven Twitter Content as a PDF or spreadsheet by using “save as” or similar functionality. Exporting Twitter Content to a datastore as a service or other cloud based service, however, is not permitted”
Seems to me that a SQL db would constitute a “datastore”, and that this datastore is used to drive a “service”.
Pzelnip, you’re right. Seems that lately tweeter frowns upon storing the tweets in a database, at least as as permanent store. Anyway, when I first contacted them about two years ago, they told me that the technique described in the article was ok, or at least no reason for a ban. So, I guess that the spirit of the norm is that they don’t want people making tools to analyze past tweets using streaming API.
Thank you so much for your comment.
hi! I tried to run your code but I’m getting this error:
Warning: fopen(http://…@stream.twitter.com/1/statuses/filter.json) [function.fopen]: failed to open stream: No connection could be made because the target machine actively refused it. in E:\wamp\www\twitter\app\demo\twitter_watch\watch.php on line 17
Is it possible to increase the 60 second timeout on the watch.php file? I let it run, but get a Fatal error: Maximum execution time of 60 seconds exceeded in …/twitterwatch/watch.php on line 27
So I’ve got everything working great, only problem is I can’t figure out how to stop to the watch/stream. I had to actually change the configuration files to a invalid username and password, then kill the process on the mySQL server.
). We need to decouple the reading of the stream from twitter and the serving to our clients because we can only have a single active instance of the twitter streaming api at a given time, and we’re going to leave the listening proccess on for a long time using a daemon/service approach. On top of that, we want to store the tweets in a database for later perusal and data mining.
32 Responses to Twitter alerts: using twitter streaming API for fun and profit
plotti
October 29th, 2009 at 21:32
Very nice example, I like how you explained things simply and easy.
Do you know of a way to do twitter searches that go back more than 7 days?
David Pardo
October 30th, 2009 at 11:17
Thank you, plotti. Twitter has stated that they’re keeping the tweets, but they’re not available yet over the API. I’ll try to keep you informed if that changes.
Magda
November 30th, 2009 at 06:58
Wow. This is possibly the clearest explanation of streaming API for Twitter on the web.
I’m trying to decipher how to do this in real-time in Flash, but I might just do it your way.
Thank you.
Jed Herzog
January 26th, 2010 at 19:07
This is the best tutorial on the Twitter Streaming API for PHP out there. Awesome job. I was a little disappointed to get to the end, have working code, and then find out that you don’t think this is ready for production. What do you think it would take to get his code ready for production? What are the issues? Have you looked at phirehose before?
I am trying to use the twitter streaming API for a personal project. If you think you could help me for a reasonable amount of compensation please contact me.
David Pardo
January 26th, 2010 at 19:14
Hi Jed,
I don’t think it’s ready for production since it doesn’t take care of disconnections/reconnections, neither can update the stream for new filter words. Anyway, I’ve already used it with minor modifications for some customers. I’ve been trying phirehose lately and looks great, but can’t vouch for it yet.
If you want me to help you with a project, drop me a line to david@corunet.com with your idea and I will try to send you a budget.
Dan Goodwin
February 28th, 2010 at 22:24
That is a very helpful, very well written article. Thanks for taking the time to write it up and share.
Ispe
April 26th, 2010 at 10:31
David
Thank you sooooooo much – You rock
arsyan
August 18th, 2010 at 20:32
Hi there, great article, i refer to your code a lot. Now im trying to figure out one thing, instead of the new tweet appears at the bottom and the top vanishes, how do i reverse that? mening to say newer tweets will show at top and the bottom one vanishes like tweetdec style.
I tried renaming first to last last to first but doesnt seem to work. Is it more complicated to start from top of issit very simple that i couldnt see it yet.
If u can kindly guide on how to reverse the tweets would be great. Thanks!
A.G.
September 24th, 2010 at 11:49
Hi there,
“Text: This is the twetter status. 140 chars max”
Thats just what the user is allowed to enter. If you take a look at the message, it contains of a lot more characters. That’s because links etc. are send as html.
I’m not at home so I cant give you the max number of what I have counted so far
JS
December 21st, 2010 at 06:35
Great tutorial! I have one question: in logic.js, when calling renderTweet(), you pass the function a second argument, ‘hidden’. I see what it achieves by inspecting the DOM in Firebug, but how does this take effect in your code? it’s a neat trick, but could you explain it? Thanks!
aci cartagena
January 31st, 2011 at 13:58
used your code as part of a mashup, between twitter and google maps. thankyou! will credit you in the about page.
cmaciasg
February 23rd, 2011 at 21:37
Hola david,
Me alegra ver que se hacen cosas majas en España también, he estado buscando información sobre el Streaming API de Google y es de los mejores tutoriales que he encontrado, mis felicitaciones.
Una pregunta, podemos hacer algo parecido sin tener que guardar datos en base de datos? es decir, mostrarlo a través de web según nos llegan de cURL?
Entiendo que en este caso no tendría sentido el php, no? Entiendo que el php no es algo que se quede “abierto” ejecutando algo, sería mas javascript o se te ocurre algo?
Muchas gracias y enhorabuena otra vez.
Rico
June 25th, 2011 at 20:24
Hi
Thx for your amazing tutorial…
Actually when I launch your script nothing appears… (mySQL OK, Server OK..)
Has it had some change on twitter API side or the script should still work?
Thx for your help
Rico
Peter Richardson
August 1st, 2011 at 23:41
Quick note to Chrome users: you may have to modify storing_tweets_in_the_database.php to see the results of the flush(). Here’s my solution:
http://stackoverflow.com/questions/6001628/php-flush-not-working-in-chrome/6904643#6904643
Steve
September 2nd, 2011 at 14:37
Fantastic, well written post about connecting to Twitter API. I have spent the afternoon playing around with this and I’ve learned a lot. No problems so far. Thank you.
Steve
September 2nd, 2011 at 14:39
Also, a question – how can I get the frontend to keep going continuously even if i’m not running storing_tweets_in_the_database.php ?
Rico
October 3rd, 2011 at 12:19
Hi all !
thx for this amazing post !
It worked until last Friday, because of the SSL only access to the stream of twitter API…. (here : https://dev.twitter.com/blog/streaming-api-turning-ssl-only-september-29th)
Do you have a simple solution for non expert in SSL developer in php ? to make this code working again ?
Thx
Rico
Javier Jaramillo
October 12th, 2011 at 23:09
Rico: as the dev.twitter.com URL says, you just have to change the ‘http’ by ‘https’.
If it ain’t working yet (as happened to me yesterday on this windows machine), then its because of configuration/setup.
Check that the: php_openssl extension is loaded and that you set allow_url_fopen = On in your php.ini
That should do it.
Good luck.
Al
October 15th, 2011 at 12:55
Great article!
One question. What syntax should I use to combine predicates eg. add a track and a location? I’m trying things like… ‘track=’twitter&locations=360,180′ but not having success.
Thanks.
Al
Felipe Signorini
October 31st, 2011 at 21:03
Awesome, this tutorial is pratice and good. Really a like to see de continuos posts, phirehose….
Jose Manuel
November 13th, 2011 at 23:57
Briliant, very interesting and easy! mi admiración por ocmpartirlo.
Rico, to make it work with https, change in file watch.php, line 15:
$instream = fopen(‘http:
for
$instream = fopen(‘https:
and you are set!
cheers
Twitter Real Time Search « thewayofcode
November 30th, 2011 at 10:35
[...] Blog post that really helped me with the javascript part of the development: http://blog.corunet.com/twitter-alerts-using-twitter-streaming-api/ [...]
Pzelnip
December 14th, 2011 at 22:19
Perhaps I’m mistaken, but isn’t storing tweets into a database a violation of the Terms of Service (TOS) for the streaming API?
https://dev.twitter.com/terms/api-terms
Reads:
“You may export or extract non-programmatic, GUI-driven Twitter Content as a PDF or spreadsheet by using “save as” or similar functionality. Exporting Twitter Content to a datastore as a service or other cloud based service, however, is not permitted”
Seems to me that a SQL db would constitute a “datastore”, and that this datastore is used to drive a “service”.
David Pardo
December 14th, 2011 at 23:07
Pzelnip, you’re right. Seems that lately tweeter frowns upon storing the tweets in a database, at least as as permanent store. Anyway, when I first contacted them about two years ago, they told me that the technique described in the article was ok, or at least no reason for a ban. So, I guess that the spirit of the norm is that they don’t want people making tools to analyze past tweets using streaming API.
Thank you so much for your comment.
Mark
December 19th, 2011 at 15:56
hi! I tried to run your code but I’m getting this error:
Warning: fopen(http://…@stream.twitter.com/1/statuses/filter.json) [function.fopen]: failed to open stream: No connection could be made because the target machine actively refused it. in E:\wamp\www\twitter\app\demo\twitter_watch\watch.php on line 17
what could be the possible cause of this?
Michael
February 1st, 2012 at 23:37
Is it possible to increase the 60 second timeout on the watch.php file? I let it run, but get a Fatal error: Maximum execution time of 60 seconds exceeded in …/twitterwatch/watch.php on line 27
David Pardo
February 2nd, 2012 at 00:34
Try http://php.net/manual/en/function.set-time-limit.php
David Pardo
February 2nd, 2012 at 00:35
Mark, remember to use https instead of http
Jake
February 10th, 2012 at 04:42
So I’ve got everything working great, only problem is I can’t figure out how to stop to the watch/stream. I had to actually change the configuration files to a invalid username and password, then kill the process on the mySQL server.
Thanks for the help.
Simon
February 26th, 2012 at 04:13
I did exactly as the instructions said and downloaded the code. But, I find that it is not working.
Alberto
March 9th, 2012 at 18:03
Why you didn’t use the Consumer key, Consumer secret, Access token and Access token secret?
Thank you
David Pardo
April 11th, 2012 at 01:13
Hi Alberto,
The code predates the new access guides for a couple of years… XD