Whether you use UNIX/Linux or Windows, upgrading minor versions of MySQL is simpleinstall the new version as if the other version does not exist.
By the Way
Before upgrading to a new minor version of MySQL, back up your existing databases.
MySQL AB has made it easy to upgrade within the same base version; 5.0 is a base version, 5.1 is a base version, and so on. This means that upgrading any minor version in the 5.0.x family is as simple as installing the new software on top of the old software. With the Windows Installer, this is all invisible to you but is exactly what occurs. UNIX/Linux users who install from the binary distribution will simply be re-linking the "mysql" symbolic directory to the new, unpacked distribution, as part of the installation process.
If you run into problems during the upgrade process, refer to the troubleshooting tips at http://dev.mysql.com/doc/refman/5.0/en/upgrade.html. However, upgrading minor versions of MySQL has always been a painless experience for me, on multiple platforms.