Really simple script to consume the firehose. Basically it will allow you to connect and stream the data down to the terminal in a prettier format. It aslo allows you to slow down how fast it reads firehose or how many messages it downloads from the firehose.
#Setup This script requires the requests library see the installation instructions for how to install.
Basic usage
python firehoser.py -a [ACCESS_TOKEN] -u [FIREHOSE_URL]
Slowing down requests to be half a second appart
python firehoser.py -a [ACCESS_TOKEN] -u [FIREHOSE_URL] -s 0.5
Only downloading n number of messages
python firehoser.py -a [ACCESS_TOKEN] -u [FIREHOSE_URL] -n 10
Access tokens can be obtained in a few ways the developer conection has instructions for obtaining an access token.
All configuration can be done either as command line parameters or as parameters as a configuration file. Command line parameters will override those parameters in the config file.
The format of the config file is this.
[firehose]
access_token=138256612637...
firehose_url=https://firehose1.omniture.com/api/1/stream/jgrover-geometrixx
By default the script will read sample.conf unless overridden by a command line file
The OAuth access token.
Command Line Usage
python firehoser.py -a 138256612637...
Config File
access_token=138256612637...
The complete URL for your firehose stream
Command Line Usage
python firehoser.py -u "https://firehose1.omniture.com/api/1/stream/jgrover-geometrixx"
Config File
firehose_url=https://firehose1.omniture.com/api/1/stream/jgrover-geometrixx
Number of seconds between each read (can be a decimal)
Command Line Usage
python firehoser.py -s 0.5
Config File
seconds=0.5
Defaults to 0
Number of messages to download
Command Line Usage
python firehoser.py -n 10
Config File
number=10
Defaults to 0
A relative URL reference to a custom config file. Allows you to have multiple configs
Command Line Usage
python firehoser.py -c "my_config.conf"
Defaults to sample.conf