You do need to restart, but I would suggest to use:
kill -USR1 `cat /var/run/httpd.pid`
as this will gracefully restart the server, in other words, apache will kill all of the child servers that are not serving clients, and have them reload with the new config, and when the childs that are serving clints are done, they will be killed and regenerated also.
In some other words, it restarts apache without ANY downtime, UNLESS you have an error in you http.conf file, and apache will die....