One Nation and the federal opposition are “dehumanising” and “scapegoating” immigrants while drawing on a “deep vein of racism”, Australia’s federal race discrimination commissioner says.

Giridharan Sivaraman made the comments as part of a panel discussion at a Brisbane seminar on human rights, hosted by the state’s human rights commission.

Asked what his priority issues were, Sivaraman said the country was facing “a very pronounced political fault line”.

“On one side of that fault line you have two parties, a populist party and now the federal opposition, who are dehumanising, scapegoating migrants,” he said on Wednesday.

Together, they encouraged people to blame migrants for everything from the housing crisis and traffic to the cost of milk, he said.

“That is implicitly racist, because … the only signal as to whether someone is a migrant usually is the colour of their skin, or their name, or their accent, and it draws on this deep vein of racism that’s always existed from the time of colonisation” he said.

“This notion that some people belong here more than others or some people are superior to others.”

Announcing a new policy on immigration last month, the opposition leader, Angus Taylor, said “many” migrants would be a “net drain” on Australia and announced that his approach would be to “discriminate based on values”.

Immigration was also a centrepiece of his budget reply speech last week. He proposed reducing overall numbers by using a cap based on housing construction and banning non-citizens from schemes like jobseeker, the age pension, and the NDIS.

“This is about mass migration running ahead of the homes, roads, hospitals, schools and services Australia can provide,” Taylor said in the speech to parliament.

Sivaraman said that over the next two years, he expected an escalation in “the racism that accompanies” blaming immigrants, posing a “significant issue” for his work.

“The only way we’ll actually deal with that – I think that racism, effectively – is to build class solidarity … between black, brown, and white people,” he said.

“Because until you can convince a white worker that it’s in their interest to combat racism – that racism is actually one of the causes of their inequity – you’re not going to build solidarity”.

In response to the comments, Taylor said: “This is the government’s failure, not that of migrant communities.

“We think migration is incredibly important to this country. It always has been and it always will be.

“But the numbers have been too high and standards to low. The numbers got up to 550,000 in a year and housing construction was going backwards.

“It’s no wonder young Australians can’t get into a home. Labor is not only exceeding their own migration targets every year, they’re not even meeting their housing targets. They’re building about 70,000 houses a year under their target.”

A spokesperson for Senator Pauline Hanson denied that the party was racist or populist or had dehumanised or scapegoated migrants.

“We don’t have a problem with migrants, we don’t, and we’re not anti-migration,” he said.

“We advocate lower immigration to reduce demand for housing in this country.”