Section 23.1. Web Services

23.1. Web Services

There are many Web services and applications on the Net, providing a wide variety of services. You will find application programmer interfaces (APIs) from most of the big players today, i.e., Yahoo!, Google, eBay, and Amazon, to name a few. In the past, APIs have been used just to access data using these services; however, today's APIs are different. They are rich and fully featured, and you are able to actually integrate services into your own personal Web sites and Web pages, commonly known as "Mash-ups."

This is an area of active interest that we will continue to explore (REST, XML, RSS, etc.), but for now, we are going to take a trip back in time to play around with an older interface that is both useful and has longevity, the stock quote server from Yahoo! at http://finance.yahoo.com.

23.1.1. Yahoo! Finance Stock Quote Server

If you visit the Web site and pull up a quotation for any stock, you will find a Uniform Resource Locator (URL) link under the basic quote data labeled "Download Data," which lets users download a CSV file suitable for importing into Microsoft Excel or Intuit Quicken:

http://finance.yahoo.com/d/quotes.csv?s=GOOG&f=sl1d1t1c1ohgv&e=.csv

If your browser's MIME settings are set correctly, your browser will actually launch Excel with the resulting data. This is due primarily to the final variable (key-value) pair found in the link, e=.csv. This variable is actually not used by the server as it always sends back data in CSV format anyway.

If we use our friend urllib.urlopen(), we see that for any stock ticker symbol, one CSV string is returned:

>>> from urllib import urlopen >>> u = urlopen('http://quote.yahoo.com/d/     quotes.csv?s=YHOO&f=sl1d1t1c1ohgv') >>> for row in u: ...    print 'row' ... '"YHOO",30.76,"5/23/     2006","4:00pm",+0.30,31.07,31.63,30.76,28594020\r\n' >>> f.close()

The string would then have to be manually parsed (by stripping the trailing whitespace and splitting on the comma delimiter). As an alternative to parsing the data string ourselves, we can use the csv module, introduced in Python 2.3, which does both the string split and the whitespace strip. Using csv, we can replace the for loop above with the following assuming all other lines are left intact:

>>> import csv >>> for row in csv.reader(u): ...     print row ... ['YHOO', '30.76', '5/23/2006', '4:00pm', '+0.30',       '31.07', '31.63', '30.76', '28594020']

By analyzing the argument field f passed to the server and from reading Yahoo!'s online help for this service, you will see that the symbols (sl1d1t1c1ohgv) correspond to: ticker symbol, last price, date, time, change, open price, daily high, daily low, and volume.

You can get more information by checking the Yahoo! Finance Help pagesjust search for "download data" or "download spreadsheet format." Further analysis of the API reveals a few more options such as the previous closing price, the percentage change of the current price to the previous close, the 52-week high and low, etc. All in all, the options can be summarized in Table 23.1 along with the formats of the returned components.

Table 23.1. Yahoo! Finance Stock Quote Server Parameters

Stock Quotation Data	Field Name[a]	Format Returned[b]
Stock ticker symbol	s	"YHOO"
Price of last trade	l1	328
Last trade date	d1	"2/2/2000"
Time of last trade	t1	"4:00pm"
Change from previous close	c1	+10.625
Percentage change from previous close	p2	"+3.35%"
Previous closing price	p	317.375
Last opening price	o	321.484375
Daily high price	h	337
Daily low price	g	317
52-week range	w	"110 - 500.125"
Volume for the day	v	6703300
Market capitalization	j1	86.343B
Earnings per share	e	0.20
Price-to-earnings ratio	r	1586.88
Company name	n	"YAHOO INC"

^[a] First character of field name is alphabetic; the second, if any, is numeric.

^[b] Some values returned quoted although all are returned in one CSV string.

The field names are given in the order you want them from the server. Just concatenate them together as a single argument to the field parameter f as part of the requesting URL. As mentioned in the returned value footnote, some of components returned are quoted separately. It is up to the parser to properly extract the data. Observe the resulting (sub)strings when parsed manually vs. using the csv module in our example above. If a value is not available, the quote server returns "N/A" (separately quoted since that field is, which makes it consistent ... a good thing).

For example, if we give the server a field request of f=sl1d1c1p2, we get a string like the following returned for a valid stock ticker:

"YHOO",166.203125,"2/23/2000",+12.390625,"+8.06%"

For the case where the stock is no longer publicly traded, we get something like this instead (note again how fields that come back quoted still do, even if N/A):

"PBLS.OB",0.00,"N/A",N/A,"N/A"

The quote server will also allow you to specify multiple stock ticker symbols, as in s=YHOO,GOOG,EBAY,AMZN. You will get back one row of data like the above for each company. Just keep in mind that "[any] redistribution of quotes data displayed on Yahoo! is strictly prohibited," as quoted in the Yahoo! Finance Help pages, so you should only be using these data for personal reasons. Also be aware that all of the quotes you download are delayed.

Using what we know now, let us build an example (Example 23.1) application that will read and display some stock quote data for some of our favorite Internet companies.

Example 23.1. Yahoo! Finance Stock Quote Example (stock.py)

This script downloads and displays stock prices from the Yahoo! quote server.

1  #!/usr/bin/env python 2   3  from time import ctime 4  from urllib import urlopen 5 6  ticks = ('YHOO', 'GOOG', 'EBAY', 'AMZN') 7  URL = 'http://quote.yahoo.com/d/quotes.csv?s=%s&f=sl1c1p2' 8 9  print '\nPrices quoted as of:', ctime() 10 print '\nTICKER'.ljust(9), 'PRICE'.ljust(8), 'CHG'.ljust(5), '%AGE' 11 print '------'.ljust(8), '-----'.ljust(8), '---'.ljust(5), '----' 12 u = urlopen(URL % ','.join(ticks)) 13 14 for row in u: 15     tick, price, chg, per = row.split(',') 16     print eval(tick).ljust(7), \ 17               ('%.2f' % round(float(price), 2)).rjust(6), \ 18               chg.rjust(6), eval(per.rstrip()).rjust(6) 19 20 f.close()

If we run this script, we will get output that looks like the following:

$ stock.py Prices quoted as of: Sat May 27 03:25:56 2006 TICKER  PRICE    CHG   %AGE ------  -----    ---   ---- YHOO    33.02  +0.10 +0.30% GOOG   381.35  -1.64 -0.43% EBAY    34.20  +0.32 +0.94% AMZN    36.07  +0.44 +1.23%