Tony Burke says authorities “know the state of mind” of each of the 34 Australian women and children stuck in a Syrian detention camp, but says his options to prevent them returning to Australia are limited.

The home affairs minister, who represents a south-western Sydney electorate with a high Muslim population, also warned Pauline Hanson’s recent derogatory comments against Muslims in Australia could incite violence.

The Australian chapter of Islamist group Hizb ut-Tahrir could also soon be banned after Burke confirmed he was in the process of adding it to a list of hate groups under legislation passed in January.

_{Sign up: AU Breaking News email}

Under the new laws, it is a crime to associate with, recruit, train, or provide support for a designated group, with an individual found guilty of “intentionally” directing the activities of a listed hate group facing up to 15 years in jail.

Eager to draw attention away from its own internal troubles, the Coalition this week has intensified its criticism of the federal government’s response to the Australians remaining in the increasingly unstable and violent detention camp.

Eleven women and 23 Australian children remain in Roj camp in north-eastern Syria near the Turkish and Iraqi borders, after a failed attempt to reach Damascus on Monday. They are the wives, widows and children of slain or jailed Islamic State fighters – many of whom arrived in Syria in the mid-2010s.

One woman was issued with a temporary exclusion order this week, banning her entry to Australia for up to two years.

Burke told ABC’s Insiders on Sunday the cohort was “not consistent” in their ideas and beliefs.

“I can give the complete confidence to community [that] we know the different individuals; we know the state of mind and the effective ideology of different individuals – they are not a coherent cohort,” he said

The minister added he had “complete confidence” in his intelligence agencies conducting the risk assessments for the group.

“The fact that one person has been pulled out for saying that person meets the threshold for a temporary exclusion order is because, quite specifically, of what we know about that individual.”

Under the Passport Act, Australian citizens are entitled to a passport subject to limited restrictions, such as the person being the subject of an arrest warrant.

Burke said the government was “actively making sure we do nothing to help them”.

“Other than a temporary exclusion order, there isn’t a legislative power to be able to stop an Australian citizen from entering Australia,” he said.

The shadow defence minister, James Paterson, told Sky News the women in the group should face charges for travelling to join an Islamist caliphate against advice and laws.

“Every one of them should face charges if they ever find their way back home, but frankly, the priority should be keeping them offshore where they can do no harm to Australia,” Paterson said.

On Thursday the shadow attorney general, Michaelia Cash, claimed Burke was deliberately choosing not to use further exclusion orders because he lacked “political courage”.

“Minister Burke has the power. He has the legal authority. He has the publicly available facts. He has the low threshold designed for exactly this purpose,” Cash said in a statement.

“What minister Burke lacks is the political courage to use the law to protect Australians.”

On Sunday, Burke hit back at Cash, questioning whether she was “fit” to hold a shadow ministry role.

“One of my concerns with how the opposition have handled this is they’ve effectively said the minister should be able to make it up,” he said.

“Anyone who has that sort of view is not fit to handle the nation’s security.”

Guardian Australia reported the group of women and children are facing dramatically deteriorating conditions with their tents demolished and their possessions seized this week.

The camp, run by Kurdish officials but expected to be transferred to the Syrian government, could soon be closed.

Asked what the government would do for the group should the camp close, Burke responded: “Well, the people will be in an intolerable position, entirely through actions they themselves took, entirely through shocking decisions that those parents made.”

Burke also turned to comments made by Hanson earlier this week, who he said was trying to bait the Muslim population of Lakemba into lashing out at her.

The One Nation senator prompted sharp criticism after claiming on a Sky News opinion program last week she did not believe there were any “good Muslims”. Federal police confirmed on Friday they received “reports of a crime” related to Hanson’s comments.

Burke said the last time Hanson had visited Lakemba, she had been welcomed with warmth and hospitality rather than abuse or protest.

“I think what is happening here is part of Pauline Hanson’s frustration with Lakemba is … it didn’t give her what she wanted,” he said.

But he warned comments, like Hanson’s, risked triggering more violence and disunity in Australia.

“It’s not just the cruelty of [her comments]. There’s a national security angle here as well,” Burke said.

“I just say to people: don’t pretend to care about national security and then make it harder for our agencies, and more likely that violence will occur.”