Applications that are compiled with header files from and linked to one version of glibc are intended to continue to work with later versions of glibc. This backward compatibility generally means that you do not have to rebuild applications just because a new version of glibc has been released.
There are practical limitations to this backward compatibility. First, mixing objects linked against different versions of glibc in a single executable may sometimes accidentally work, but not by intentional design. Note that this includes dynamically loaded objects as well as statically linked objects. Second, the application should be standard-conforming; an application that depends on side effects of bugs or on unspecified behavior in one version of glibc may not continue to work with later versions of glibc, and an application that links to private symbols in glibc (generally speaking, symbols prefixed with a _ character) is also very unlikely to work with later versions of glibc.
One way that this backward compatibility is maintained is with versioned symbols. When the authors of glibc wish to introduce an incompatible change in glibc, they preserve the original implementation or write a compatible implementation of the interface in question and mark it with the older glibc version number. They then implement the newer, different interface (which may differ in semantics, signature, or both) and mark it with the new glibc version number. Applications built against the older glibc version use the older interface; applications built against the newer glibc version use the new interface.
Most other libraries maintain compatibility by including the version number in the library name and having multiple different versions installed at the same time if necessary; for example, GTK+ 1.2 and GTK+ 2.0 might both be installed on the same system, each with its own set of header and library files and with the version name embedded in the path to the header files and in the names of the library files.
Part of the standard for naming shared libraries on Linux includes a major version number, in order to provide multiple versions of a library on a system. This is not much used for its intended purpose, because it does not allow you to link new applications against multiple versions of a library on a single system; it merely allows backward compatibility to be maintained for already-built applications built on older systems. In practice, developers have found that they have needed to build applications against multiple versions of the same library on the system, and therefore most major libraries have migrated to providing the version number of the library as part of the name of the library.