If you right click on the expression you can delete. I am going to assume if the expression is used in other formulas you can't delete it until all of the where referenced is deleted. It should be greyed out at the bottom of your list.
I right clicked my heart out before posting the question in the first place. I've attached a simple file created long ago with expressions that were declared but not applied. Right clicking on the expression in the expression editor or the part navigator does not offer the option to delete it. I get the same result in a brand new file. This is NX 11.0.0.33 MP0, by the way.