Vue 3: withDefaults() prevent TypeScript from checking conditional props

Question

I've tried to create conditional props for my <Button /> component using discriminated union types. I don't want to allow pass down props like to, hash, linkBehavior when another prop called render-as is set to "button" (and allow it only if render-as="link"). I've also set up default value of renderAs to button using withDefaults().

Problem: When I try to set up props like to, hash, linkBehvaior and I intentionally omit to set up render-as (because it has its default value). I expect that TS throws an error, because of incompatibility of props. And it doesn't. (If I set render-as manually, then everything works perfectly.) Is there any solution to fix this and still keep my default value renderAs: 'button'? Thank you.

Button.vue

<script setup lang="ts">

export type ButtonCommonProps = {
  priority: 'primary' | 'secondary'
  size: 'large' | 'small'
  // ...
}

type ButtonConditionalProps =
  | {
    renderAs: 'button'
    to: never
    hash: never
    linkBehavior: never
  }
  | {
    renderAs: 'link'
    to?: string
    hash?: string
    linkBehavior?: '_self' | '_blank' | '_parent' | '_top'
  }

type ButtonProps = ButtonCommonProps & ButtonConditionalProps

const props = withDefaults(defineProps<ButtonProps>(), {
  renderAs: 'button',
  to: undefined,
  hash: undefined,
  linkBehavior: undefined,
   // ...
})
// rest of the component...

Another Vue file

<template>
  <!-- Here I expect error and it really occurs -->
  <Button priority="primary" size"small" render-as="button" to="https://example.com">
    Some text
  </Button>

  <!-- Here I expect error as well and it won't occur -->
  <Button priority="primary" size"small" to="https://example.com">
    Some text
  </Button>
</template>