Mucked around with it a bit more
seems that no matter what I change, old GFX card, new one, no card, CD drives, no CD drives / peripherals. Nothing I change makes any difference now. If the HDD is plugged in it wont power on, if its unplugged, it will.
Looks like a good old-fashioned HDD...