Hack 53. Fetching Files from the Web | Tivo Hacks. 100 Industrial-strength Tips Tools 2003

Hack 53 Fetching Files from the Web

Now that TiVo has high-speed Internet access, let's download some files from the Web .

While we now have our TiVo online [Hack #49] and can telnet to the box [Hack #52], there's still no good way to move files to and from TiVo without doing so via the serial interface using rz/sz [Hack #36].

While rz/sz [Hack #36] will happily move files over the serial line, they don't play nicely with non-8-bit connections like the telnet interface.

Let's think about this for a bit. How does TiVo itself download stuff? Those software updates, for instance. It sets up a dialup-PPP connection to TiVo, Inc. and downloads files over HTTP. Why, then, can't we simply use the same mechanism over our higher-class permanent network connection? Well, we can.

TiVo sports a little application called http_get , which, as its name suggests, grabs files remotely using HTTP as its transport mechanism. One caveat is that http_get can't do any domain name resolution.

On top of requiring the IP address, http_get is actually quite finicky . Let's fiddle a bit and see what we can learn, shall we?

Let's say, for instance, you wanted to download a copy of the TiVoWeb server [Hack #65]. You'd invoke http_get like so:

 bash-2.02#  http_get -T 0 -C 0 -D /var/hack -U http://199.240.141.102:80/ tivoweb-tcl-1.9.4.tar.gz

This command asks TiVo to download the file at http://199.240.141.102:80/tivoweb-tcl-1.9.4.tar.gz ( -U is for URL) to the local /var/hack [Hack #36] directory ( -D is for directory). Because http_get was originally designed for the TiVo to use exclusively when connecting to TiVo, Inc., it also requires the - T and the - C flags, representing the serial number of the unit and the ID of the call, respectively. Simply set both to . Be sure to specify the URL fully. You'll get the following error if you leave off the :80 specifying the default HTTP port number:

 connect failed, reason = Connection refused Segmentation fault

You'll get the following error if you try to specify a DNS name instead of an IP address:

 connect failed, reason = Network is unreachable Segmentation fault

Not being able to specify a DNS name turns out to be more problematic than you'd think. Many web servers do something called "virtual hosting" these days. This means that many domain names actually map to the same IP address, since specifying the host name specified onboard is the only way to tell two requests for two different domains apart. Take oreilly.com and oreillynet.com , for examplefor both, one of their IP addresses is 209.201.239.37. The server at that IP address knows which site you mean to visit only because your browser passes along the domain name portion of the URL as part of the HTTP request. If you don't specify a host name, the web server is either going to pick one or attempt to redirect youthe latter being something http_get doesn't like at all.

About the only way around this shortcoming is to download the file you're after to a machine that is accessible by IP addressone on your local network or a server out there on the Internetand instruct TiVo to grab it from there. Of course, you can still pass files back and forth serially [Hack #36] as a last resort.

Top