C2924XL trunk port connected to server

[mb2]

We have a switch used for connecting client servers to our backup server - Historically all ports have been in the one vlan.
Is it possible to have ports clients are connected to in seperate vlans, and the port that connects to the backup server in trunk mode and still allow access to the backup server? (i.e. so clients would not be able to 'see' other clients servers, but could still access the backup server)

All servers would be in the same subnet (192.168.5.0/24)

Example:

!
interface FastEthernet0/20
switchport access vlan 2
spanning-tree portfast
!
interface FastEthernet0/21
switchport access vlan 3
spanning-tree portfast
!
!
interface FastEthernet0/22
switchport access vlan 4
spanning-tree portfast
!
interface FastEthernet0/23
switchport access vlan 5
spanning-tree portfast
!
!
interface FastEthernet0/24
switchport trunk encapsulation dot1q
switchport mode trunk
!

switchport trunk allowed vlan all was also enabled on FE0/24

I have tried the above without success.
If I place FE0/24 trunk native vlan 3, then client on FE0/21 can successfully access the Backup Server.

Any assistance is greatly appreciated.

Regards,
MB