What type of firewall have you compiled? OPEN, CLOSED, UNKNOWN or custom. I'll assume you have open and are using IPFW. To block a port, say Telnet 21, to all outside traffice but 2 subnets 1.2.3.4 and 5.6.7.8 to your box 2.5.4.1, add these rules:
ipfw add 700 allow tcp from 1.2.3.4/24 to 2.5.4.1 23 via xl0
ipfw add 710 allow tcp from 5.6.7.8/24 to 2.5.4.1 23 via xl0
ipfw add 720 deny tcp from any to 2.5.4.1 23 via xl0
Of course you can use whatever ruleset number you use instead of 700, 710, and 720 as well as use your NIC in place of xl0.
Hope this helps.