Anthony Albanese has claimed he was heckled by people at one of Australia’s largest mosques because they were unhappy his government has outlawed “extremist organisations” like Hizb ut-Tahrir, although he failed to provide any evidence for his claim.

The prime minister and the home affairs minister, Tony Burke, were at Lakemba mosque in Sydney’s west on Friday morning before celebrations for the end of Ramadan, one of the most sacred dates in the Islamic faith.

Albanese sat at the front of the mosque, listening to the Lebanese Muslim Association secretary, Gamel Kheir, read out a statement panning Australia’s involvement in the Middle East war and the effect it has had on the Muslim community, when some worshippers began to protest against his and Burke’s attendance.

Some were heard yelling phrases including “disgrace”, “shame” and “genocide supporters”. One man who booed Albanese was led out by police. He was released without charge after being given a move-on direction.

Albanese made a hasty exit, avoiding a large crowd waiting for him on the mosque’s steps.

At a press conference on Friday afternoon, Albanese said there were 30,000 people in Lakemba, and the reception to his visit was “overwhelmingly … incredibly positive”.

“There were a couple of hecklers inside, they were dealt with,” he said. “Contrary to what’s been suggested, no one was rushed out, we just sat there, it was dealt with. It was dealt with by the community themselves because overwhelmingly they did not want that to occur.”

“I thank them for the very warm reception that occurred. Yes, there were a couple of people who were heckling, some people don’t like the fact that we have outlawed extremist organisations like Hizb ut-Tahrir, and that brought a response from a couple of people.”

Guardian Australia analysis of videos taken of the heckling found no reference to the group. The prime minister’s office has been contacted for comment.

The Lebanese Muslim Association defended its decision to host Albanese at the mosque, saying the prime minister’s attendance was not a “betrayal” of its position on Gaza and Lebanon.

“We are aware of the reaction to the prime minister’s attendance and we understand that emotions are high, particularly given the ongoing suffering in Gaza and the devastation in Lebanon,” it said in a statement. “These are not distant issues for our community. They are deeply personal, and they shape how many people feel in this moment.

“But we also need to be clear. Choosing to engage with the elected leadership of this country is not a betrayal of those concerns. It is how we give them a voice. Walking away from engagement has not advanced our community, nor has it changed outcomes overseas. It has not reduced Islamophobia here, and it has not strengthened our ability to influence decisions that affect us.”

“When leaders enter our mosque, they do so in the presence of a community that is informed, engaged and unafraid to speak plainly. We do not open our doors for appearances. We do so because real conversations require access, and because our community deserves to be heard directly, not spoken about from a distance.”

The association wrote that its “position on Gaza and Lebanon is clear and unchanged”.

“We continue to stand with those suffering, to call for justice, and to advocate for an end to violence and the protection of innocent lives,” the statement said. “Engaging with government does not weaken that position. It strengthens our ability to express where it matters the most.”

It comes as Israel carries out air and artillery strikes in Lebanon and pushes on with a ground invasion in a campaign against Hezbollah. Israel’s assault on Lebanon has killed at least 912 people, including 111 children, and wounded 2,221 others, according to the Lebanese health ministry, with more than 1 million people displaced.

The federal government last week announced it would send a specialist surveillance aircraft and stocks of air-to-air missiles to the UAE to protect Australians in the Gulf state, which has been targeted by Tehran in retaliation for the US and Israel’s bombings of Iran.

About a year ago, Burke was forced to leave a Muslim prayer event in Lakemba, which is part of his Watson electorate, due to security concerns after protesters turned up intent on telling the local MP he was “not welcome”.

Protesters planned to heckle and “interrupt” Burke when he was due to address congregants at nightly Ramadan prayers in Lakemba on 21 March last year. However, he left the venue before the prayers were completed without addressing the congregation.

Bilal Rauf, a barrister and senior adviser to the Australian National Imams Council, said many felt distressed and frustrated with how the government had dealt with the community.

He cited Albanese’s decision to invite Israeli president Isaac Herzog to Australia in the wake of the Bondi terror attack, and the police response to Muslim men praying during a protest over Herzog’s visit.

“So many people in our society are connected to what’s happening around the world, and they are hurting. They are seeing their family members killed. They are seeing homes of family burned,” he said.