Soz you got the probs.
But yep the drill is, go along with them ( cos they can simply hit you with an ASBO, which has a small level of proof and if you breach it, you get a criminal record). But note time and place and conversation and names and numbers , if possible ( but noting time and place 'should' mean the person can be tracked and identified).
Then contact the relevent authority ( email will do), advice can be got here. Be it Chief Constatble or council. I good 1 is to simply write to your local councillor, stating the facts...they love to do work for their constituents and with luck will do all the work for you.
I did that with a gruff nasty security in front of bournemouth pier, result...nice letter from council saying they have no health and safty probs with us! Oh as an aside, 1 week after the inncident, in exactly same place, same day, same time...a cop car drove up to us, cop lent out and shouted at us to do tricks!!! So there are no rules for whats right or wrong i fear
More recently, in Portsmouth, we were asked to stop cos we were 'dissin Nelson'. Yep I know e been dead 200 years..but some1 objected to us bouncing near a statue of him.
We stopped, but we related this to a JJ member who is a bar manager...and knows the right people in the council who give permission for 'outdoor stuff'. Yep, yet another nice letter saying we can bounce all over pompy!! And we had to use it a couple of weeks later.
Basically go along with them...then get , basically, legal justification....if they cant, you get a letter saying its OK for you to carry on.