What would you do in this situation? Living in sheltered accommodation scheme.?

Don't over react and don't take it so personally. They are just failing tounderstand that you have aproblem. And it seems that it will be fairly easy to solve so do your best to forget about it. Its really not worth falling out with people or losing sleep.