I have observed interoperability problems when the client calls the StartNewKeyPairRequest and requests the privateKeyFormat format as PFX (and retrieves it using FinishRequest).

PFX is a container, and can contain just the private key. And that is indeed what OpenSSL-based implementations of GDS will do/are doing currently. They return the PFX which contains only the private key (which is correct per the UA spec).

There are, however, following problems with such PFX:
1) I found no way of processing/opening/importing it in .NET (both /.NET Framework and .NET 6+) , leaving aside writing the code to parse it out "manually".
2) Less importantly, Windows cannot view the contents of such PFX either (right-click -> Open).

What works for (1) and (2) above is when the PFX also contains the certificate itself.

I suggest that the spec is changed to require the PFX to contain both the private and the certificate.