The problem you are having is that you are trying to define "average velocity" in more than one way, but those different definitions are not actually compatible.
For example, you can claim "a tiger is a big cat with sharp teeth and black stripes" and also claim "a tiger is also a grey mammal with a long nose," but then it will be confusing when one "tiger" eats plants and another "tiger" eats meat.
In your first paragraph you say "$v = \frac{ds}{dt}$" is average velocity. Then a few sentences later you say "$v = s/t$" is average velocity. These are two different possible definitions. Do they match? You see for yourself that they do not match.
And why should they match? Here are three other possible definitions of "average velocity," none of which give the same result
- $v = \frac{v(0)+v(t)}{2}$
- $v = \frac{v(0)+v(t/2)+v(t)}{3}$
- $v = \frac{s}{2t}$
So how do we decide which one should be the real definition of "average velocity"? We choose the definition that is the most useful in the most cases. Historically, the definition that has turned out to be the most useful is the definition that satisfies two rules:
- When the velocity is constant the average velocity should match the constant velocity.
- If you know the average velocity over a time period $t$ you should be able to calculate the displacement with the simple equation $s = v_{avg}t$
Now once we've decided on that definition of average velocity we need to stick with it, and we shouldn't be surprised when other calculations don't give the same result. We also shouldn't be surprised if there are other useful things to know about velocity. "Average velocity" tells us a specific thing about velocity, but it doesn't tell us everything and so it's helpful to have other concepts like "instantaneous velocity" to give us information that "average velocity" doesn't.