Managing RDP through Group Policy

[DeP]

The only weakness that I have heartburn over is that the remote Windows PC can memorize the RDP password

You should be able to turn this off here:
Computer Configuration\Administrative Templates\Windows Components\Terminal Services\Client