Scrapping penalty rates will cost some Victorian workers as much as $428 a week, the Victorian government believes.

The federal government has ruled out changes to penalty rates, but Victoria’s unions say the government’s word means little.

“They’ve made a promise not to cut penalty rates – is that like a promise not to cut health?” Victorian Trades Hall Council campaigner, Wil Stracke, told reporters on Thursday.

“The fact that they’ve promised not to do it makes us even more worried.”

Premier Daniel Andrews, said his government’s submission to the Productivity Commission’s review of workplace relations would show how many Victorians rely on penalty rates.

“If the Abbott government changes penalty rates, hundreds of thousands of Victorian workers will be worse off,” Andrews said.

A Victorian machinery operator could lose up to $428 a week, while a retail employee could lose up to $300 a week.

Andrews said 374,000 Victorians who usually work on Saturdays in their main job would be worse off, along with 213,000 who work on Sundays.

Stracke said workers deserved to be compensated for missing out on family time, and they relied on the money.

“These workers rely on these rates to pay their bills, pay their mortgages,” she said.

A spokesman for Opposition Leader, Matthew Guy, said he supported penalty rates.

Federal employment minister, Eric Abetz, insisted penalty rates and the minimum wage won’t be targeted by the coalition government.

But it was up to the Productivity Commission to decide what it looked at in its review of the industrial relations system.

“I’m not going to … try to tell the commission how it should or should not interpret the language of the terms of reference,” the minister told a Senate estimates hearing in Canberra on Thursday.

AAP