Eric Pickles has refused to back down over his decision to allow West Watford allotments to be built on as part of the health campus scheme, ahead of a High Court showdown.
The Secretary of State of Communities and Local Government was challenged to undo the decision after the plot-holders’ legal team obtained fresh evidence about the case.
However, according to court documents, Mr Pickles declined saying he had no power to undo the decision and did not accept it was flawed.
The move means the battle over the 100-year-old allotments is set go to the High Court for a judicial review next week.
The plot-holders are challenging permission Mr Pickles gave Watford Borough Council in December to use the allotments in the regeneration scheme which aims to build up to 750 home and new businesses behind Vicarage Road.
The scheme also promises to leave space for potential new hospital facilities as well.
The allotment-holders’ skeleton argument, which was submitted to the judge ahead of the case, said Mr Pickles decision was based on a "simply wrong" assumption that the scheme could not go ahead without using the allotment land.
The allotment-holders’ barrister, Jason Coppel QC said there needed to be exceptional circumstances for the Secretary of State sign off on the destruction of allotments in use.
Mr Pickles justified his Farm Terrace decision saying he had seen evidence that the health campus would not be "economically viable" without the allotment land.
And this could deprive the public of the scheme’s benefits, which included "improved hospital facilities".
However Mr Coppel said this was not supported by the council’s evidence, which said the allotments would make the scheme "more viable"
He added: "In other words that with the allotments, the project would be more likely to make a profit."
Mr Coppel said immediately after Mr Pickles’s decision there were a number of developments that further undermined the basis of his reasoning.
These included the release of the health campus master plan, which showed that up to 20 per cent of the allotment land would be used for a Watford FC car park.
Another development was the Environment Agency approving a flood plain plan that would allow homes to be built on a wider area of the regeneration site.
Mr Coppel argued: "As a result, it was clear by the end of June 2014, even if the Secretary of State’s second decision had once appeared to be defensible, any such defence had fallen away".
He added that at the very least Mr Pickles should retake the decision based on the correct factual basis.