None of the answers so far has addressed the underlying question, which is why it seems that multiplication is asymmetric and satisfies "positive times positive is positive" while "negative times negative is positive". The reason is that multiplication is not as simple as it seems.
$\def\nn{\mathbb{N}}$
$\def\zz{\mathbb{Z}}$
$\def\rr{\mathbb{R}}$
This answer is necessarily long to explain in reasonable detail how one builds real arithmetic intuitively from scratch, rather than purely by axiomatization. This is unavoidable since the field axioms can show that $-1 \times -1 = 1$ but give no insight whatsoever into why it should be like this.
Viewpoint 1 is sort of the more fundamental and relies on less intuition, but you will need to follow slowly and carefully in order to completely grasp and apprciate it, so if you want a quick answer skip to Viewpoint 2 and stop once I begin to tie it to Viewpoint 1.
Viewpoint 1
First we can start with the natural numbers $\nn$, which you intuitively understand, whether using blocks or using strings of symbols, or treating each natural number as a way to represent the number of times of repetition of some procedure.
Real numbers $\rr$ represent displacement, as you mentioned in your question. But what does $a \times b$ represent? You cannot multiply displacement by another displacement! So something else is meant here. Let's see what we can define intuitively. Define $k \times x$ to be "$k$ times as much as $x$" for any $k \in \nn$ and $x \in \rr$. Of course $0_\nn \times x = 0_\rr$ and $1_\nn \times x = x$, for any $x \in \rr$. Note that I explicitly distinguish between $0_\nn$ ($0$ times) and $0_\rr$ (zero amount). Also note that $x \times k$ is not assigned any meaning (yet), since it makes no sense to talk about "$x$ as much as $k$ times".
Note that $\nn$ embeds into $\rr$ in the following manner. You can treat $k_\rr$ as $k \times 1_\rr$, which by our definition means "$k$ times of $1_\rr$". In the physical world this corresponds to the amount "$k$ units".
Now you can verify intuitively that two crucial properties hold:
$( a +_{_\nn} b ) \times x = a \times x + b \times x$ for any $a,b \in \nn$ and $x \in \rr$.
$( a \times_{_\nn} b ) \times x = a \times ( b \times x )$.
Here I've used "$+_{_\nn}$" and "$\times_{_\nn}$" for addition and multiplication in $\nn$ to distinguish it from "$+$" and "$\times$" in $\rr$. Based on these two properties, we say that $\nn$ acts on $\rr$ via $\times$. This notion of action is ubiquitous one, even if you don't realize it. For example, $\nn$ acts on physical entities via $\def\of{\text{ times }}$$\of$ as follows: "$k \of X = \text{$k$ instances of $X$}$". We say "$2$ apples" by which we mean "$2 \of apples$" = "2 instances of apples". We intuitively understand that we can handle abstract numbers to perform counting of physical entities, which is why we have:
$( 2 + 3 ) \of apples = 2 \of apples + 3 \of apples$.
$( 2 \cdot 3 ) \of apples = 2 \of ( 3 \of apples )$.
[likewise for all other physical entities, which is the key insight; it works not only for apples!]
Similarly, $\nn$ acts on (real-world) actions via $\of$ as follows: "$k \of A = \text{$A$ repeated $k$ times}$". Again we can check that both properties hold.
All these are examples of the action of $\nn$ on some collection. Some of these can be extended meaningfully to action of $\zz$ on the same collection, but not others. What we would want is that $(-_{_\zz}k) \times x = -( k \times x )$ for any $k \in \zz$ and $x \in \rr$. For displacement this would mean that "$(-1_\zz) \times x$" is simply "the opposite displacement to $x$".
Already here we can see the asymmetry, because $(-1_\zz) \times (-1_\rr) = 1_\rr$ and $1_\zz \times 1_\rr = 1_\rr$. The former means "$-1$ times the opposite to unit displacement", while the latter just means "unit displacement", and both are the same. So the answer to your question under this viewpoint is that the asymmetry comes from the concept of iteration that is intrinsic to the action of $\zz$ on $\rr$.
Just for completeness, $\zz$ does not act on physical entities, because there are no direct opposites to physical entities that when combined with the original yields nothing. Using the concept of "lack" or "debt" simply changes the collection; originally it is the collection of physically existing entities, subsequently it is the collection of records of relative quantities of physical entities. It is essentially the same way we construct $\zz$ from $\nn$ mathematically. Also $\zz$ only acts on reversible actions.
Viewpoint 2
Real numbers are scale factors, or better still the scalings themselves with centre at the origin. $2$ is the $2$-times zoom on the origin. $1$ is the identity scaling that does nothing at all. $0$ is the scaling that collapses everything into the origin. $-1$ is the scaling that takes every point to the point opposite the origin. Multiplication of real numbers is the same as composing the two scalings together (doing one after another). Note that intuitively the order of scalings does not matter, giving us commutativity of multiplication in $\rr$.
Obviously $-1 \times -1 = 1$, answering the question of asymmetry. The underlying reason is actually the same as in the other viewpoint, because here positive is special simply because $1$ is the identity scaling (keep it as it is), while there positive is special because $1$ is the unit amount (exactly as it is). Notice the core similarity!
Now the above only explains the multiplicative structure of the reals, so where does the additive structure come in? We can in fact combine it with Viewpoint 1 by considering the real numbers themselves as positions on a line through the origin (equivalently displacements from the origin), and then define that a scaling $r$ is precisely the scaling centred at $0$ on the line that brings the point $1$ on the line to the point $r$. Here we have used the intuition that there is such a scaling and that it is unique, so there is a one-to-one correspondence between the scalings and points on the line. To keep compatible with the above paragraph, we still define a composed scaling $r \times s$ as the composition scaling by $r$ then by $s$.
Since the scaling $( r \times s )$ brings the point $x$ to the same point if we did first scaling $s$ and then $r$, we do have the second property for action of scalings $\rr$ on points $\rr$. This is clearer in symbols, where we write $\def\on{\text{ on }}$"$r \on x$" to denote "the result of scaling $r$ applied on $x$":
$( r \times s ) \on x = r \on ( s \on x )$.
From the definition of scaling and commutativity of scaling, we get:
$r \times x = ( r \times x ) \on 1 = r \on ( x \on 1 ) = r \on x$.
$r \on s = r \times s = s \times r = s \on r$.
We now define addition of points $x,y$ on the line to be vector addition (add the displacements from the origin), and then define addition of scalings $r,s$ to be exactly the same according to the correspondence between scalings and points! Note that by intuition (or 'definition' of scaling/zoom) we can see that the result of vector addition is the same no matter what scale we are observing it from, and so scaling before adding should give the same as adding before scaling. Symbolically:
$r \on x + r \on y = r \on ( x + y )$.
From the above results we can finally get the first property for action of $\rr$ on $\rr$:
$( r + s ) \on x = x \on ( r + s ) = x \on r + x \on s = r \on x + s \on x$.
These properties correspond to the properties of multiplication of real numbers (commutativity, associativity and distributivity over addition), which are the most difficult to justify intuitively. The rest are left as an exercise.
Therefore we have established that arithmetic of $\rr$ captures facts about scaling (or zooming) in the real world, at least according to our intuition. Even if it turns out that it is not exactly true, it certainly serves as a good approximation, and hence investigating theorems about $\rr$ give us approximate facts about the real world.
Footnotes
Of course the above viewpoints cannot be used to prove that real numbers exist, but at least give a reasonable justification for why we believe $\rr$ is useful in describing the real-world. Nevertheless, the explanation can be converted into a rigorous construction of the real numbers in any formal system that is powerful enough to permit construction of simple collections, functions from $\nn$ to $\nn$, equivalence relations and equivalence classes.
