Dredd -
you can use Group Policy to get alot of what you're after, I think. Group Policy has a set of administrative templates that are alternatives to editing the registry directly.
1) Run gpedit.msc (you have to be logged on as an administrator)
2) Expand User Configuration, then expand Administrative Templates.
You'll see folders labelled:
"Start Menu & Taskbars"
"Desktop"
"Control Panel" and
"System"
among others.
You can then configure various policies to restrict access or hide things from users. For example, in "Start Menu & Taskbars" you can disable changes to the Taskbar and Start Menu settings, or you can remove specific items from the Start menu.
Under "Desktop" you can disable adjusting the desktop toolbars, disable dragging, dropping and closing the Toolbar's tools, and tell the system not to save any settings at exit.
Under Control Panel, you have various options about what control panel apps are available to users - including none. And under System, you can limit what programs users can run.
One other thing you could do - visit your favorite local computer retailer and ask them what they've done to protect the systems they've set out for public display. May save you some time -