In this chapter, we overviewed the various ways people use to improve the accuracy performance of the content-based information retrieval system. The gap between low-level features and high level semantics has been the main obstacle for developing more successful retrieval systems. It can be expected that it will still remain as one of the most challenging research topics in this field.