Archiving Twitter

Twitter data is only available via the search API for up to 7 days. Data for a given account only goes back a few thousand tweets. Thus archiving tweets can be a useful activity. This entry details a few options and in the process shows some neat tips and tricks for pulling down data.

Using twarc

twarc is a powerful command line tool and Python library for archiving Twitter JSON data. You will need to obtain a free API key from Twitter in order to start archiving tweets.

Using Javascript and the DataHub

See https://github.com/OKFN-BR/BusaoSP/blob/master/getdata.js