Whether you use Linux/Unix or Windows, upgrading minor versions of MySQL is quite simpleinstall the new version as if the other version does not exist.
By the Way
Before upgrading to a new minor version of MySQL, you should back up your existing databases.
MySQL AB has made it quite easy to upgrade within the same base version; 4.0 is a base version, 4.1 is a base version, and so on. This means that upgrading any minor version in the 4.0.x family is as simple as installing the new software on top of the old software. With the Windows installer, this is all invisible to you, but is exactly what occurs. Linux/Unix users who install from the binary distribution will simply be re-linking the "mysql" symbolic directory to the new, unpacked distribution, as part of the installation process.
If you run into problems during the upgrade process, there are troubleshooting tips at http://www.mysql.com/doc/en/Upgrade.html. However, upgrading minor versions of MySQL has always been a painless experience for me, on multiple platforms.