I'm guessing that the issue is that the size of the bar is changing instead of the chart, and you would prefer to keep the bar size more consistent. If this is the case, I think one alternative is to add the chart to different sections, e.g., RF_a, RF_b, etc. and then conditionally suppress the sections based on a range of groups, e.g., one section might have a formula like this in the conditional suppression area for an RF section:
distinctcount({table.groupfield}) < 5 or
distinctcount({table.groupfield}) > 10
-LB