I think people with holiday homes should pay full rate. I know they would argue that they don't use the services- bin collections etc, but the extra that they pay could help to support services for those that are deprived of a home because the prices have been driven out of the reach of local workers.
I had a freind who rented a cottage in a tiny yorkshire village called Thwaite for about a year. It had been bought by it's owner as a holiday let, but it was cold and damp and they struggled to let it for holidays. There were about 40 cottages in the village- all holiday homes bar about 5. Pretty as a picture, dead as a doornail. I'm sure 30 years ago it would have been a thriving community too.