64 Checking for Overdue Books at the Library
#64 Checking for Overdue Books at the Library
Most of the lynx - related scripts in this book are built around either passing information to a web server via a method=get form transmission (the passed information is appended to the URL, with a ? separating the URL and its data) or simply scraping information from predefined web page content. There's a third category of page, however, that uses a method=post form transmission for submitting information from the web browser to the remote server.
While more difficult, this method can also be emulated using lynx , as this script shows. This specific script sends a data stream to the Boulder (Colorado) Public Library website, logging the specified user in and extracting a list of books and other items checked out, with due dates. Notice in particular the creation and use of the postdata temporary file.
The Code
#!/bin/sh # checklibrary - Logs in to the Boulder Public Library computer # system and shows the due date of everything checked out for # the specified user. A demonstration of how to work with the # method="post" form with lynx. lib1="http://nell.boulder.lib.co.us/patroninfo" lib2="items" libacctdb="$HOME/bin/.library.account.info" postdata="/tmp/$(basename#!/bin/sh # checklibrary - Logs in to the Boulder Public Library computer # system and shows the due date of everything checked out for # the specified user. A demonstration of how to work with the # method="post" form with lynx. lib1="http://nell.boulder.lib.co.us/patroninfo" lib2="items" libacctdb="$HOME/bin/.library.account. info " postdata="/tmp/$(basename $0).$$" awkdata="/tmp/$(basename $0).awk.$$" # We need: name cardno recordno # Given the first, look for the other two in the library account database if [ $# -eq 0 ] ; then echo "Usage: $(basename $0) \"card holder\""; exit 0 fi acctinfo="$(grep -i "$1" $libacctdb)" name="$(echo $acctinfo cut -d: -f1 sed 's/ /+/g')" cardno="$(echo $acctinfo cut -d: -f2)" recordno="$(echo $acctinfo cut -d: -f3)" if [ -z "$acctinfo" ] ; then echo "Problem: account \"$1\" not found in library account database." exit 1 elif [ $(grep -i "$1" $libacctdb wc -l) -gt 1 ] ; then echo "Problem: account \"$1\" matches more than one record in library db." exit 1 elif [ -z "$cardno" -o -z "$recordno" ] ; then echo "Problem: card or record information corrupted in database." exit 1 fi trap "/bin/rm -f $postdata $awkdata" 0 cat << EOF > $postdata name=${name}&code=${cardno}&submit=Display+record+for+person+named+above EOF cat << "EOF" > $awkdata { if ( NR % 3 == 1) { title=$0 } if ( NR % 3 == 2) { print $0 "" title } } EOF lynx -source -post-data "$lib1/$recordno/$lib2" < $postdata \ grep -E '(^<td name =\"renew)' \ sed 's/<[^>]*>//g' \ awk -f $awkdata sort exit 0).$$" awkdata="/tmp/$(basename#!/bin/sh # checklibrary - Logs in to the Boulder Public Library computer # system and shows the due date of everything checked out for # the specified user. A demonstration of how to work with the # method="post" form with lynx. lib1="http://nell.boulder.lib.co.us/patroninfo" lib2="items" libacctdb="$HOME/bin/.library.account. info " postdata="/tmp/$(basename $0).$$" awkdata="/tmp/$(basename $0).awk.$$" # We need: name cardno recordno # Given the first, look for the other two in the library account database if [ $# -eq 0 ] ; then echo "Usage: $(basename $0) \"card holder\""; exit 0 fi acctinfo="$(grep -i "$1" $libacctdb)" name="$(echo $acctinfo cut -d: -f1 sed 's/ /+/g')" cardno="$(echo $acctinfo cut -d: -f2)" recordno="$(echo $acctinfo cut -d: -f3)" if [ -z "$acctinfo" ] ; then echo "Problem: account \"$1\" not found in library account database." exit 1 elif [ $(grep -i "$1" $libacctdb wc -l) -gt 1 ] ; then echo "Problem: account \"$1\" matches more than one record in library db." exit 1 elif [ -z "$cardno" -o -z "$recordno" ] ; then echo "Problem: card or record information corrupted in database." exit 1 fi trap "/bin/rm -f $postdata $awkdata" 0 cat << EOF > $postdata name=${name}&code=${cardno}&submit=Display+record+for+person+named+above EOF cat << "EOF" > $awkdata { if ( NR % 3 == 1) { title=$0 } if ( NR % 3 == 2) { print $0 "" title } } EOF lynx -source -post-data "$lib1/$recordno/$lib2" < $postdata \ grep -E '(^<td name =\"renew)' \ sed 's/<[^>]*>//g' \ awk -f $awkdata sort exit 0).awk.$$" # We need: name cardno recordno # Given the first, look for the other two in the library account database if [ $# -eq 0 ] ; then echo "Usage: $(basename#!/bin/sh # checklibrary - Logs in to the Boulder Public Library computer # system and shows the due date of everything checked out for # the specified user. A demonstration of how to work with the # method="post" form with lynx. lib1="http://nell.boulder.lib.co.us/patroninfo" lib2="items" libacctdb="$HOME/bin/.library.account. info " postdata="/tmp/$(basename $0).$$" awkdata="/tmp/$(basename $0).awk.$$" # We need: name cardno recordno # Given the first, look for the other two in the library account database if [ $# -eq 0 ] ; then echo "Usage: $(basename $0) \"card holder\""; exit 0 fi acctinfo="$(grep -i "$1" $libacctdb)" name="$(echo $acctinfo cut -d: -f1 sed 's/ /+/g')" cardno="$(echo $acctinfo cut -d: -f2)" recordno="$(echo $acctinfo cut -d: -f3)" if [ -z "$acctinfo" ] ; then echo "Problem: account \"$1\" not found in library account database." exit 1 elif [ $(grep -i "$1" $libacctdb wc -l) -gt 1 ] ; then echo "Problem: account \"$1\" matches more than one record in library db." exit 1 elif [ -z "$cardno" -o -z "$recordno" ] ; then echo "Problem: card or record information corrupted in database." exit 1 fi trap "/bin/rm -f $postdata $awkdata" 0 cat << EOF > $postdata name=${name}&code=${cardno}&submit=Display+record+for+person+named+above EOF cat << "EOF" > $awkdata { if ( NR % 3 == 1) { title=$0 } if ( NR % 3 == 2) { print $0 "" title } } EOF lynx -source -post-data "$lib1/$recordno/$lib2" < $postdata \ grep -E '(^<td name =\"renew)' \ sed 's/<[^>]*>//g' \ awk -f $awkdata sort exit 0) \"card holder\""; exit 0 fi acctinfo="$(grep -i "" $libacctdb)" name="$(echo $acctinfo cut -d: -f1 sed 's/ /+/g')" cardno="$(echo $acctinfo cut -d: -f2)" recordno="$(echo $acctinfo cut -d: -f3)" if [ -z "$acctinfo" ] ; then echo "Problem: account \"\" not found in library account database." exit 1 elif [ $(grep -i "" $libacctdb wc -l) -gt 1 ] ; then echo "Problem: account \"\" matches more than one record in library db." exit 1 elif [ -z "$cardno" -o -z "$recordno" ] ; then echo "Problem: card or record information corrupted in database." exit 1 fi trap "/bin/rm -f $postdata $awkdata" 0 cat << EOF > $postdata name=${name}&code=${cardno}&submit=Display+record+for+person+named+above EOF cat << "EOF" > $awkdata { if ( NR % 3 == 1) { title=#!/bin/sh # checklibrary - Logs in to the Boulder Public Library computer # system and shows the due date of everything checked out for # the specified user. A demonstration of how to work with the # method="post" form with lynx. lib1="http://nell.boulder.lib.co.us/patroninfo" lib2="items" libacctdb="$HOME/bin/.library.account. info " postdata="/tmp/$(basename $0).$$" awkdata="/tmp/$(basename $0).awk.$$" # We need: name cardno recordno # Given the first, look for the other two in the library account database if [ $# -eq 0 ] ; then echo "Usage: $(basename $0) \"card holder\""; exit 0 fi acctinfo="$(grep -i "$1" $libacctdb)" name="$(echo $acctinfo cut -d: -f1 sed 's/ /+/g')" cardno="$(echo $acctinfo cut -d: -f2)" recordno="$(echo $acctinfo cut -d: -f3)" if [ -z "$acctinfo" ] ; then echo "Problem: account \"$1\" not found in library account database." exit 1 elif [ $(grep -i "$1" $libacctdb wc -l) -gt 1 ] ; then echo "Problem: account \"$1\" matches more than one record in library db." exit 1 elif [ -z "$cardno" -o -z "$recordno" ] ; then echo "Problem: card or record information corrupted in database." exit 1 fi trap "/bin/rm -f $postdata $awkdata" 0 cat << EOF > $postdata name=${name}&code=${cardno}&submit=Display+record+for+person+named+above EOF cat << "EOF" > $awkdata { if ( NR % 3 == 1) { title=$0 } if ( NR % 3 == 2) { print $0 "" title } } EOF lynx -source -post-data "$lib1/$recordno/$lib2" < $postdata \ grep -E '(^<td name =\"renew)' \ sed 's/<[^>]*>//g' \ awk -f $awkdata sort exit 0} if ( NR % 3 == 2) { print#!/bin/sh # checklibrary - Logs in to the Boulder Public Library computer # system and shows the due date of everything checked out for # the specified user. A demonstration of how to work with the # method="post" form with lynx. lib1="http://nell.boulder.lib.co.us/patroninfo" lib2="items" libacctdb="$HOME/bin/.library.account. info " postdata="/tmp/$(basename $0).$$" awkdata="/tmp/$(basename $0).awk.$$" # We need: name cardno recordno # Given the first, look for the other two in the library account database if [ $# -eq 0 ] ; then echo "Usage: $(basename $0) \"card holder\""; exit 0 fi acctinfo="$(grep -i "$1" $libacctdb)" name="$(echo $acctinfo cut -d: -f1 sed 's/ /+/g')" cardno="$(echo $acctinfo cut -d: -f2)" recordno="$(echo $acctinfo cut -d: -f3)" if [ -z "$acctinfo" ] ; then echo "Problem: account \"$1\" not found in library account database." exit 1 elif [ $(grep -i "$1" $libacctdb wc -l) -gt 1 ] ; then echo "Problem: account \"$1\" matches more than one record in library db." exit 1 elif [ -z "$cardno" -o -z "$recordno" ] ; then echo "Problem: card or record information corrupted in database." exit 1 fi trap "/bin/rm -f $postdata $awkdata" 0 cat << EOF > $postdata name=${name}&code=${cardno}&submit=Display+record+for+person+named+above EOF cat << "EOF" > $awkdata { if ( NR % 3 == 1) { title=$0 } if ( NR % 3 == 2) { print $0 "" title } } EOF lynx -source -post-data "$lib1/$recordno/$lib2" < $postdata \ grep -E '(^<td name =\"renew)' \ sed 's/<[^>]*>//g' \ awk -f $awkdata sort exit 0"" title } } EOF lynx -source -post-data "$lib1/$recordno/$lib2" < $postdata \ grep -E '(^<td name=\"renew)' \ sed 's/<[^>]*>//g' \ awk -f $awkdata sort exit 0
How It Works
To get your own version of this script working with your own public library (or similar system), the basic technique is to browse to the page on the system website at which you must submit your account information. In the case of this script, that page is http://nell.boulder.lib.co.us/patroninfo . Then, on that page, use the View Source capability of your browser to identify the names of the form input elements into which you must submit your account information. In the case of this script, the two input text elements are name and code (library card number). To duplicate that, I have stored the required information in the $postdata file:
name=${name}&code=${cardno}&submit=Display+record+for+person+named+above I then use this information to populate the input elements by passing the information to lynx :
lynx -source -post-data "$lib1/$recordno/$lib2" < $postdata
The account information used in the temporary $postdata file, as well as in other places in the script, is stored in a shared database library called .library.account.info , which you must build by hand. The toughest part of building this account database was identifying the internal library ID of my account, but again, the View Source capability of a modern browser is all that's needed: I just logged in to the library database itself with my name and card number and then looked at the source code of the resultant page. Buried in the data was the line
<A HREF="/patroninfo/12019/items"
Voil ! I then stored my internal ID value, 12019, in the library account information database file.
Finally, the awk script makes the output prettier:
if ( NR % 3 == 1) { title= if ( NR % 3 == 1) { title=$0 } if ( NR % 3 == 2) { print $0 "" title } } if ( NR % 3 == 2) { print if ( NR % 3 == 1) { title=$0 } if ( NR % 3 == 2) { print $0 "" title } "" title } It joins every second and third line of the output, with the first line of each discarded, because it's not necessary for the desired output information. The end result is quite readable and attractive.
Running the Script
To run this script, simply specify a pattern that uniquely identifies one person in the library account database on your machine. My account database looks like the following:
$ cat ~/.library.account.info # name : card number : library internal ID Dave Taylor:D0060681:12019
| Special note | In the interest of not blasting my library card number throughout the known universe, the data file shown for this script is not exactly correct. Therefore, you won't be able to run the script and find out what books I have checked out, but the general concept is still informative. |
The Results
It's a simple matter to see what's due:
$ checklibrary Dave DUE 09-06-03 Duke the lost engine/ W. Awdry ; DUE 09-06-03 Farmer Will / Jane Cowen-Fletche DUE 09-06-03 Five little monkeys jumping on t DUE 09-06-03 Five little monkeys sitting in a DUE 09-06-03 Main line engines/ W. Awdry ; wi DUE 09-06-03 Now we can have a wedding! / Jud DUE 09-06-03 The eagle catcher/ by Margaret C DUE 09-06-03 The summer cat/ story and pictur DUE 09-06-03 The tempest : [a novel] / Juan M
Hacking the Script
There are further levels of sophistication that can be added to this script, the most useful of which is to compare the date string values for today, tomorrow, and the following day with the due dates in the script output to enable warnings of books due in the next few days.
Another useful addition is a wrapper that can be called from cron to automatically email the results of the checklibrary script on a schedule. This is also easily done:
#!/bin/sh # booksdue - Emails results of checklibrary script. checklibrary="$HOME/bin/checklibrary" results="/tmp/results.$$" to="taylor@intuitive.com" trap "/bin/rm -f $results" 0 $checklibrary Dave > $results if [ ! -s $results ] ; then exit 0 # no books checked out! fi ( echo "Subject: Boulder Public Library - Books Due" echo "To: $to" echo "From: (The Library Scraper) www@intuitive.com" echo "" cat $results ) sendmail -t exit 0
Notice that if no books are checked out, the script exits without sending any email, to avoid annoying "no books checked out" kinds of messages.
