The party alleges the Great Yarmouth MP made “threats of physical violence” against its party chair.