The answer helped a bunch but commenter was correct that old card wasn't deleted.

Assuming you would only ever have one card for a customer you would do this instead:

//get customer
$customer = \Stripe\Customer::retrieve($client_id);

//get the only card's ID
$card_id=$customer->sources->data[0]->id;
//delete the card
$customer->sources->retrieve($card_id)->delete();

//add new card
$new_card = $customer->sources->create(array("source" => $token));
$customer->default_source = $new_card->id;
$customer->save();

The answer helped a bunch but commenter was correct that old card wasn't deleted.

Assuming you would only ever have one card for a customer you would do this instead:

//get customer
$customer = \Stripe\Customer::retrieve($client_id);

//get the only card's ID
$card_id=$customer->sources->data[0]->id;
//delete the card if it exists
if ($card_id) $customer->sources->retrieve($card_id)->delete();

//add new card
$new_card = $customer->sources->create(array("source" => $token));
$customer->default_source = $new_card->id;
$customer->save();