NOTE: this video has no audio, but details the workflow to delete an assignment.
To delete one or more assignments for a single activity, select one or all assignments in the left column, click Edit in the upper right, select the radio button for Delete Assignment(s) and then click Delete.