Thirty organisations representing victims of violence against women and girls (VAWG) have written to the justice secretary, David Lammy, urging him to drop plans to significantly reduce the number of jury trials.

The groups said that the proposals, which will affect court cases in England and Wales, will deepen mistrust in the justice system among victims and distract from measures designed to reduce offending.

The signatories, which include Rights of Women, the End Violence Against Women Coalition, Women for Refugee Women and various branches of Women’s Aid, added that they were “deeply concerned that the curtailment of jury trials risks unfair outcomes that undermine justice for everyone”.

The letter, which has also been signed by Fiona Rutherford, the chief executive of the law reform charity Justice, said those working against violence were particularly concerned for women and girls who were “unjustly criminalised” as a result of their abuse, some of whom have faced trial themselves.

Another signatory, the Centre for Women’s Justice, has long campaigned on the issue, saying that about 70% of women in prison or under probation supervision are known to be victims of domestic abuse. It argues that some domestic abuse victims charged with criminal offences have acted under duress or in self-defence.

The controversial courts and tribunals bill was introduced in an attempt to tackle a backlog of cases in crown courts, which now stands at 80,000. The government claims this could rise to 200,000 by 2035 if action is not taken.

If the bill becomes law, jury trials would be scrapped for cases likely to carry a custodial sentence of fewer than three years. Magistrates would be granted increased sentencing powers and lone judges could hear some crown court cases.

The proposed changes would affect what are known as “either way” offences, removing the right of some defendants to opt for their case to be heard by a jury in a crown court rather than by a magistrate.

The plight of rape and sexual assault victims has frequently been used as an argument in favour of new legislation. More than 13,000 are waiting for their cases to be heard and some face delays of several years.

However, the organisations have raised a number of concerns about the bill, which survived a vote in the Commons last week. It passed with a majority of 101, although 10 Labour MPs rebelled and many more abstained.

The letter says: “VAWG organisations have long called for radical action to address the harms the criminal justice system causes for survivors, including the crown court backlog.

“However, to position the rights and interests of survivors as directly opposed to those of defendants oversimplifies the reality of how our criminal justice system treats women and survivors on all sides of our adversarial model.

“This includes a failure to recognise that women who are survivors of violence are routinely and unjustly criminalised, particularly when they are members of minoritised communities.

“We are deeply concerned that the curtailment of jury trials risks unfair outcomes that undermine justice for everyone.

“For centuries, trial by one’s peers has acted as a democratic safeguard, functioning as a vital connection between society and the law, so that justice is not determined by a single class or authority alone.”

During a debate on the proposed legislation, the Labour MP Charlotte Nichols accused the government of using rape victims as a “cudgel” to drive through changes to the justice system that may not directly benefit them.

In her speech, Nichols revealed she waited 1,088 days for her rape case to come to trial. Although she said the defendant was acquitted in a criminal court, she added that he was later ordered to pay her compensation as part of a civil case.

However, another Labour MP, Natalie Fleet, who has spoken about being groomed and raped as a teenager, said: “You know what’s worse than being raped? Facing years of waiting to see if people believe you.”

In their letter, the organisations say that any decision to reduce the number of jury trials risks “diverting resources and attention away from the substantive reform needed to address the complex, underlying causes of VAWG”.

They also raised the issue of diversity, saying that only 36% of circuit judges who sit in crown court cases are women, and 10% are from ethnic minorities.

They added that research carried out by Lammy in 2017 suggested juries did not generally discriminate against defendants on the grounds of ethnicity. However, they also cited a separate study by academic Kitty Lymperopoulou, which indicated that lone judges are more likely to give jail sentences to those from black, Asian and mixed ethnicity groups.

The letter said: “Fewer than half of victims believe they can get justice. We also know that confidence is lowest among minoritised communities … In judge-only decisions, there is increased potential for individual biases to influence the verdict.

“Conversely, the involvement of 12 randomly selected people brings a wider range of lived experiences, insights, and perspectives into the decision-making process, strengthening the fairness and balance of deliberations.

“Requiring a unanimous verdict means that every juror’s viewpoint must be considered, ensuring that the final decision reflects collective deliberation rather than relying on a single viewpoint.

“We know that black people, older people and women elect for trial at the crown court at higher rates than other groups. The protections of a jury trial are therefore central to the integrity of the system and are critical for victims, particularly women and those from minoritised groups, who are subject to discriminatory treatment and are already likely to mistrust the system.”

A Ministry of Justice spokesperson said: “No one has more lived and varied experience of the justice system than victims’ groups representing brave survivors across the country – and we wholeheartedly welcome all views as we move forward with these vital reforms.

“We had a positive and constructive meeting with a number of signatories today and look forward to working collaboratively with them on our reform package and the independent review of its implementation so we can deliver fairer and faster justice for all.”