Just asked the same question, then worked out the answer myself - you need to delete the events you currently have, then replace them (with your selected configuration) and the job alert ticked, before you press the Add button. In other words, start from scratch, select the Completed...