David Lammy has promised to cut the number of children kept in jail while they await trial by a quarter as part of an overhaul of youth justice rules that could also end lifelong criminal records for under-18s.

The justice secretary is publishing a white paper on Monday that he says will reduce the number of children ending up in jail – something he admits was his greatest fear growing up in Tottenham in the 1980s.

Arguing that prison can do lasting damage to children’s lives, Lammy, the deputy prime minister, is promising to reduce the use of custodial remand and short sentences for under-18s.

The Ministry of Justice (MoJ) will also consult on fining parents, and even sending them to jail, if they fail to keep their children away from crime.

He said: “Growing up in Tottenham in the 1980s, my biggest fear was ending up in prison. That may sound irrational, but in truth it was the fate of so many young Black boys like me.

“You saw it happen slowly at first. People missed school, got into petty trouble, started hanging around with the wrong crowd. No one stepped in to pull them back. For us, going to jail didn’t feel shocking or distant. It felt almost inevitable.

“I could have been one of them, but was fortunate to get a scholarship to a state boarding school, which gave me the route out that others never had.

“I often think: ‘There but for the grace of God go I.’ Even today, that line between a child who thrives and a child whose life falls apart is often painfully thin.”

Lammy, 53, argued that jailing children makes them more likely to offend in future. “For the most serious offences, custody will always be necessary to protect the public – that will never change,” he said.

“But for many children, even a short spell inside can do lasting damage, disrupting the most formative years of their lives and sometimes exposing them to more violence and criminal influence.”

The measures set out on Monday include spending an extra £15m a year to fund teams of people to work with children deemed to be at risk of offending, for example if they have been given an antisocial behaviour order (asbo).

Lammy is promising to cut the use of custodial remand for under-18s by 25% by the end of the parliament, with a view to ending it altogether. He will allow judges to offer a wider range of community sentences in a bid to further cut the numbers of children in jail .

Officials say these two measures should jointly reduce the number of young people in prison by 20%. Lammy is also promising to create a new offence of child criminal exploitation, which will penalise adults who encourage children to commit crimes.

His department will pilot the use of “youth intervention courts”, where judges and support workers will come up with individually tailored plans for young people.

This could include demanding that a child comply with certain health or educational requirements, while also keeping them under close monitoring to prevent repeat offending.

In addition, Lammy is launching a consultation on whether children should have to disclose criminal records for the rest of their lives, something he criticised in 2017 as part of his review into Black, Asian and minority ethnic people in the justice system.

“We will review how offences committed in childhood appear on criminal records, because a mistake made at 13 should not become a life sentence of closed doors and lost chances,” he said.

MoJ figures show 80% of prolific offenders committed their first crime as a child, while two-thirds of those released from custody reoffend within a year.

Rachel de Souza, the children’s commissioner, said: “I have consistently been clear about the need to reform the youth justice system.

“We must build an approach that keeps children safe, diverts them from crime wherever possible and prioritises meaningful behaviour change.”