If you want something that will run on a commodity PC, QNX will do this, and supports a GUI called Neutrino. Some other embedded system platforms also support graphical user interfaces, such as Wind River's Tilcon toolset for VXWorks.
IBM's OS/2 has been sold to a third party and is still marketed as eComStation. It is largely sold as a legacy platform supporting existng OS/2 software, with relatively little new development activity. However, it is perfectly capable of functioning as a general purpose desktop O/S and I've seen OS/2 in UK HSBC branches within the past few years. The alarm clock 'wait' cursor icon is quite distinctive.
Some other operating systems such as Haiku (a BeOS clone) or ReactOS (A Windows clone) have been produced by open-source development communities. In theory, ReactOS has a substantial degree of binary compatibility with Windows. Most third party software support for Haiku is based on ports of open-source applications.
If you relax the 'must run on a PC' constraint, some other reasonably 'modern' OS platforms come out of the woodwork.
IBM's I series is architecturally a fairly modern O/S, and was possibly the last major O/S done by people who had no exposure to Unix. It was originally designed as a replacement for IBM's mainframe O/S platforms and then re-branded as a minicomputer platform. It is a capable platform in many ways but does not have a native GUI, although IBM have done a pretty credible job of supporting J2EE based web applications on it.
You can actually still buy machines that will run software written for the Amiga or Acorn Archimedes. I have seen it estimated that the latter architecture actually still has a user base of about 10,000 in the UK, and the Amiga still has a large worldwide fan base. However, I suspect that there are is not a lot of new build software being developed for either platform. More recently there is also a RiscOS port for the Raspberry Pi.
Vax, Alpha and Itanium based machines will run VMS, although the Vax and Alpha are out of production and HP does not sell purpose-built itanium based workstation systems anymore. However, used hardware can be readily purchased on Ebay and HP will still provide VMS installers for it. They even have a VMS hobbyist program that is still active and will let you buy an install CD for a nominal price of about $USD30. VMS is architecturally quite different to Unix and was not designed to be compatible, although it uses X as a GUI.
Several mobile platforms can support a variety of application software. Although the dedicated ones like iPhone, Symbian or Windows Mobile are unlikely to be practical as a general purpose computing platform for various reasons. Android or other linux based platforms could in theory be used for a broader range of tasks. Theoretically, Android could be self-hosting - i.e. one could (in theory) actually port and run an Android development environment on Android and use an Android-based O/S on a general purpose workstation. Again, this might not work all that well in practice.