The following books provide useful general-purpose background knowledge and conceptual information related to modern cryptography.
The Handbook of Applied Cryptography by Alfred J. Menezes, Paul C. van Oorschot, and Scott A. Vanstone . This is also freely available online at www.cacr.math.uwaterloo.ca/hac/. This is one of the best and most comprehensive general technical references available. It introduces symmetric and asymmetric algorithms and provides excellent mathematical descriptions. Areas covered include pseudorandom number generator, symmetric and asymmetric algorithms, stream and block ciphers, hash functions, and digital signatures, along with descriptions of several important protocols and standards.
Introduction to Cryptography by Johannes A. Buchmann . This is another great book that shows how several important cryptographic techniques work and how to estimate algorithmic efficiency and security strength. This book explains the important mathematical methods of modern cryptography, but it assumes only basic mathematical knowledge on the part of the reader.
Applied Cryptography by Bruce Schneier . This book has become enormously popular, especially as a reader's first background book. It provides a nice light overview of most of the significant aspects of modern cryptography, and it comes with a set of C source files for all of the most notable algorithms, which is great for learning about cryptography from the inside out.
Cryptography: Theory and Practice , Second Edition, by Douglas Stinson . This book provides a highly readable and thorough coverage of cryptographic mathematics, including DES, RSA, one-way hash functions, pseudorandom number generator, and digital signatures.