Charities say rare spiders did not block “an entire new town”, as the prime minister suggested.