The rename idea won't work. The renamed file will still have the same internal structure.
The error you are getting is very common. It means that there are Oracle segments occupying the space that you are trying to release. Normally the right approach at this point would be to drop and recreate the objects that are taking up the extra space. I'm not sure if this would work with a temporary tablespace.
One approach that should work is to create a new temporary tablespace with the desired size. Then alter all user ids to have their temp tablespace assigned to the new tablespace. After all current sessions log out, the old temporary tablespace shouldn't be in use. At that point you can drop it.
As always when you make significant structural changes to your db, be sure to do backups before and after the change, so you can recover in case of problems.