Hack 63 Stocking Up on Financial Quotes

Keeping track of multiple stocks can be a cumbersome task, but using the Finance::Quote Perl module can greatly simplify it. And, while we're at it, we'll generate pretty graphs with RRDTOOL .

Collecting stock prices can be done using LWP [Hack #9] to download a financial site and regular expressions [Hack #23] to scrape the data, as well as always keeping a watchful eye for site design changes that could break things. But why go to the trouble when Finance::Quote (http://search.cpan.org/author/PJF/Finance-Quote/) provides a simple interface with numerous sources, such as Fidelity Investments, Trustnet, The Motley Fool, or Yahoo!?

Here's a typical bit of code that uses Finance::Quote to fetch stock prices:

 #!/usr/bin/perl use Finance::Quote; my $q = Finance::Quote->new; my $quotes = $q->fetch("nasdaq","IBM"); print "Price range: $quotes->{'IBM','year_range'}\n"; 

We create a new Finance::Quote object and fetch data with $q->fetch($market,@stocks) . In this case, we let the market point to nasdaq . Though @stocks is normally a list of desired stocks, we use just one ( IBM ). To get at the information that the module has grabbed for us, we use $quotes->{'IBM','year_range'} , which will get us the price range for the last 52 weeks:

 %  perl finance.pl  Price range: 54.01 - 90.404 

There is much more information in addition to year_range ; consult the Finance::Quote documentation for further explanation and details on which information is available from which sources. When in doubt, you can get a complete list of the available values by printing the returned $quotes structure:

 use Data::Dumper; print Dumper($quotes); 

Adding these two lines to the previous code produces the following output:

 $VAR1 = {           'IBM{avg_vol' => 7264727,           'IBM{div' => '0.64',           'IBM{ask' => undef,           'IBM{date' => '7/22/2003',           'IBM{method' => 'yahoo',           'IBM{div_yield' => '0.78',           'IBM{low' => '81.65',           'IBM{symbol' => 'IBM',           'IBM{cap' => '141.2B',           'IBM{day_range' => '81.65 - 83.06',           'IBM{open' => '82.50',           'IBM{bid' => undef,           'IBM{eps' => '3.86',           'IBM{time' => '1:40pm',           'IBM{currency' => 'USD',           'IBM{success' => 1,           'IBM{volume' => 6055000,           'IBM{last' => '81.70',           '  IBM{year_range' => '54.01 - 90.404'  ,           'IBM{close' => '82.50',           'IBM{high' => '83.06',           'IBM{net' => '-0.80',           'IBM{p_change' => '-0.97',           'IBM{ex_div' => 'May  7',           'IBM{price' => '81.70',           'IBM{pe' => '21.37',           'IBM{name' => 'INTL BUS MACHINE',           'IBM{div_date' => 'Jun 10'         }; 

The first part of the variable name ( IBM , in this case) is the stock symbol, the second part is a delimiter of some kind, and the third is the name of the data being referred to. The information we printed in our first code sample is emphasized in the preceding output as a guide.

We have the data; now it's time to start plotting it into a graph. As in [Hack #62] we use RRDTOOL (http://people.ee.ethz.ch/~oetiker/webtools/rrdtool/) to plot our data, but this time we will use a Perl interface. RRDTOOL has two Perl interfaces, but we will use the "Shared RRD module" only, as it is the most flexible of the two. The Perl interface will be very familiar to those who know the command-line interface.

To add data to stocks.rrd , for example, we would normally run this command:

 %  rrdtool update stocks.rrd N:12345  

Using the Perl interface, all we have to do is call RRDs::update , like this:

 use RRDs; RRDs::update ("stocks.rrd","N:12345"); 

Similarly, RRDs::create , RRD::graph , and others all work like their command-line counterparts. More information on the Perl bindings are available within the supplied RRDTOOL documentation.

Putting it all together in a Perl script, we use a "does this database exist?" check to see whether we should create a new database or update an existing one. Then, we get new stock figures using Finance::Quote and add them to our database using RRDs::update . To create graphs, we run once with RRDs::graph and --start -1w to create a graph for the last week, and once with -1m to graph the entire last month.

The Code

Save the following code in a file called grabstocks.pl :

 #!/usr/bin/perl -w use strict; use RRDs; use Finance::Quote qw/asx/; # Declare basic variables. my @stocks       = ('IBM','MSFT','LNUX'); my @stock_prices = (0,0,0); my $workdir      = "./stocks"; my $db           = "$workdir/stocks.rrd"; my $now          = time(  ); # if the database hasn't been created, # do so now, or die with an error. if (!-f $db) {     RRDs::create ($db, "--start", $now-1,           "DS:IBM:ABSOLUTE:900:0:U",           "DS:MSFT:ABSOLUTE:900:0:U",           "DS:LNUX:ABSOLUTE:900:0:U",           "RRA:AVERAGE:0.5:1:4800",           "RRA:AVERAGE:0.5:4:4800",           "RRA:AVERAGE:0.5:24:3000",     );     if (my $ERROR = RRDs::error) { die "$ERROR\n"; } } # now, get the quote information # for IBM, Microsoft, and Linux. my $q      = Finance::Quote->new(  ); my %quotes = $q->fetch("usa",@stocks); # for each of our stocks, check to  # see if we got data, and if so,  # add it to our stock prices. foreach my $code (@stocks) {     my $count = 0; # array index.     unless ($quote{$code, "success"}) {         warn "$code lookup failed: ".$quote{$code,"errormsg"}."\n";         $count++; next; # well, that's not a good sign.     }     # update the stock price, and move to the next.     $stock_prices[$count] = $quote{$code,'last'}; $count++; } # we have our stock prices; update our database. RRDs::update($db, "--template=" . join(':',@stocks),                   "$now:" . join(':',@stock_prices)); if (my $ERROR = RRDs::error) { die "$ERROR\n"; } # Generate weekly graph. RRDs::graph("$workdir/stocks-weekly.png",   "--title",     'Finance::Quote example',   "--start",     "-1w",   "--end",       $now+60,   "--imgformat", "PNG",   "--interlace", "--width=450",   "DEF:ibm=$db:IBM:AVERAGE",   "DEF:msft=$db:MSFT:AVERAGE",   "DEF:lnux=$db:LNUX:AVERAGE",   "LINE1:ibm#ff4400:ibm\c",   "LINE1:msft#11EE11:msft\c",   "LINE1:lnux#FF0000:lnux\c" ); if (my $ERROR = RRDs::error) { die "$ERROR\n"; } # Generate monthly graph. RRDs::graph ("$workdir/stocks-weekly.png",   "--title",     'Finance::Quote example',   "--start",     "-1m",   "--end",       $now+60,   "--imgformat", "PNG",   "--interlace", "--width=450",   "DEF:ibm=$db:IBM:AVERAGE",   "DEF:msft=$db:MSFT:AVERAGE",   "DEF:lnux=$db:LNUX:AVERAGE",   "LINE1:ibm#ff4400:ibm\c",   "LINE1:msft#11EE11:msft\c",   "LINE1:lnux#FF0000:lnux\c" ); if (my $ERROR = RRDs::error) { die "$ERROR\n"; } 

Running the Hack

First, we need a cron job [Hack #90] to run this script once every 15 minutes. To do that, add something like this to your crontab , telling cron to run our script four times every hour :

 */4 * * * Mon-Fri /path/to/your/grabstocks.pl 

With that in place, new graphs will be generated every time the script runs.

Hacking the Hack

The first and most obvious thing is to change the code to get more data for more interesting stocks. The periods chosen in this hack might also need some updating, since getting data every 15 minutes gives a much higher resolution than we need if we're interested in only monthly graphs. Likewise, running the script 24 hours a day doesn't make much sense if there will be stock changes only during business hours.

Mads Toftum