Where are my tweets?

This is a follow-up to the article I wrote about the Twitter API request limit yesterday. Besides the restricted number of requests clients are allowed to send per hour, there is another limitation that often leads to irritation: twitter sends back no more than 20 tweets per request. So if you experience missing some tweets in twhirl, this post will explain why.

Some basics

The Twitter API (Application Programming Interface) offers some methods to fetch tweets, replies or direct messages. The results are limited to twenty items, so per request, the Twitter servers will never return more than 20 tweets or direct messages.

While the requests for replies and direct messages support paging (as you know it from the website) to get the next 20 items, the request that fetches the tweets written by people you are following does not. To be fair, it did in the past, but it has been disabled for some time now.

There is no mechanism provided by the API that allows a client to find out if between two requests more than the twenty items it received were available. And even if a client would use some tricks to optimize this, fetching paged items requires additional requests, which can results in exceeding the request limit.

Why do you miss tweets in twhirl?

Sometimes users report problems that twhirl misses some of the tweets from their friends. They show up on the website, but twhirl does not fetch them. Now that you know the basics, you probably already know why this happens, and that it’s not a bug in twhirl.

Generally, when tweets are missing in twhirl, they couldn’t be fetched due to the 20 tweets per request limit. Between two requests, more than 20 items were added to the Twitter databases, but you only get the latest 20, all others are lost.

Don’t panic: most users will not miss tweets using twhirl. But if you are following a lot of users, or your friends update very often, it can happen to you. Also, if some requests to the twitter API fail due to network problems, Twitter hickups or anything else, the time between two successful requests gets quite long, and you might miss some tweets then.

How to optimize twhirl to minimize missing tweets

If you experience missing tweets, you can tweak twhirl’s connection settings. Generally, try to keep the time between refreshes as short as possible. Open the configuration dialog and switch to the connections tab.

First of all, you can increase the number of requests per hour twhirl is allowed to use for auto-refreshing your timeline. But keep in mind, the more requests you give to twhirl, the fewer you can use yourself

Then, adjust the sliders below to prioritize the type of messages you miss. If you tend to miss public tweets by your friends, give them higher priority. If you are not a frequent direct message user, give them a low priority. You will have to do some trial-and-error to find the best settings for you.

If you still miss tweets, you may consider to change which replies to see written by your friends in your twitter account settings.

Conclusion

What twhirl (and any other Twitter API client) can fetch and display to you is limited by the Twitter API. While you can optimize its settings to some extent, there is no guarantee that it will never drop a tweet.