It is 'technically' illegal in general.
The Australian Passport Office has:
Damaged passports
It is your responsibility to keep your passport intact and in good condition. Normal wear and tear will not affect its usability, but serious damage to your passport could prevent you from travelling overseas.
Contact with water or other liquids can cause serious damage. You must not tear or remove pages from your passport. It is critical that all the details and the photos on the personal data pages are legible and clear, and that there is no evidence of alteration or tampering with any aspect of the booklet.
If you are unsure whether your passport's condition is good enough for travel, you should seek advice from the Australian Passport Information Service on 131 232 or from an Australian diplomatic mission or consulate. You may need to take your passport to an Australian Passport Office for assessment.
Under the law of most countries, passports are government property. A visa that is a wet ink stamp can't change that and a sticker format is still a visa. In the rare cases where governments take back passports they have issued the visa stickers are not first removed by the government and handed back to the person to whom they were issued.
So, a passport with a stuck in visa can't be considered to remain intact once an attached visa is removed, with or without evidence of the removal.
I am not sure about Australian passports but mine, when new, only had one page of personal data. Australia mentions all the details and the photos on the personal data pages are legible and clear. I do now have personal details (name, date of birth, photographs) throughout my passport, mostly as part of self-adhesive visa stickers.
Finally, there is no evidence of alteration or tampering with any aspect of the booklet. It is not specified that any evidence must be visible to the naked eye. (However, in many cases removal of my stickers would leave evidence such as incomplete wet ink stamps, parts of signatures or other ink marks.) With modern forensic techniques it is extremely unlikely that removal (of adhesive or stapled) visas would be impossible to detect.
Clearly many people have managed to remove visas without serious consequences, though several have faced extended questioning and reprimands. But then again, many people have exceeded speed limits without facing any consequences, even though against the law.
The confusion may be that removal is a crime without an effective penalty. USA for example has a law about Destruction of Government Property but the upper limit of the sanctions imposable a fine of up to $250,000, ten years imprisonment, or both would be out of proportion and a more reasonable withdrawal of the tampered passport hardly worth the bother.
For USA there would be no point:
Do not try to remove the visa from your old passport and stick it into the new valid passport. If you do so, your visa will no longer be valid.