Hello salesclub,
As per the supported environment, CF9 is supported on Windows Server 2003. Also, you can install CF MX 6.1 and CF9 on the same server. However, did you remove the CF MX 6.1 connector before testing the same with CF9. You can't create connector to same website with different versions of ColdFusion.
Did the installation completed successfully. Were you able to access CF 9 Admin after installation. Please provide us some more information.
Regards,
Anit Kumar