In this context, an "algebra" is short for "$F$-algebra" for some field $F$. When we say $A$ is an $F$-algebra, that means the multiplication on $A$ is $F$-bilinear. In particular, this requires $F$ to be commutative (not a skew field like $\mathbb{H}$) and central (so $\mathbb{H}$ is not a $\mathbb{C}$-algebra since $Z(\mathbb{H})=\mathbb{R}$). Things go wrong if you try to loosen this definition.
Note, by the way, you can change what field you consider an algebra over. For example, $A=M_2(\mathbb{C})$ can be considered as an $\mathbb{R}$-algebra or a $\mathbb{C}$-algebra. Facts like (non)commutativity and (non)associativity are properties of $A$'s multiplication operation and do not depend on which $F$ is chosen.
Suppose $M$ and $N$ are right and left $A$-modules, respectively. Then $M\otimes_AN$ is a well defined $F$-vector space. If actually $M$ is a $(X,A)$-bimodule (meaning left $X$-module, right $A$-modules, and these module actions commute) and $N$ is an $(A,Y)$-module, then $M\otimes_AN$ will be a $(X,Y)$-bimodule. (This is only a gain in generality, not a loss, because we can take $X$ and/or $Y$ to be $F$.)
Note $\mathbb{O}$ is only an $\mathbb{R}$-algebra (unital, nonassociative), not a $\mathbb{C}$-algebra or $\mathbb{H}$-algebra. It is a left $\mathbb{C}$-vector space and a right $\mathbb{C}$-vector space (i.e. module), indeed a $(\mathbb{C},\mathbb{C})$-bimodule because it is an alternative algebra, however it is not a left or right $\mathbb{H}$-vector space (let alone a bimodule).
That said, you can still define "$\mathbb{O}\otimes_\mathbb{H}\mathbb{O}$" as a vector space to be the quotient of $\mathbb{O}\otimes_\mathbb{R}\mathbb{O}$ by the vector subspace spanned by the differences $(xh\otimes y-x\otimes hy)$ for $x,y\in\mathbb{O}$, $h\in\mathbb{H}$. Using $\ell$ for any nonzero imaginary octonion perpendicular to $\mathbb{H}$, we can say $\mathbb{O}=\mathbb{H}\oplus\mathbb{H}\ell$ is a $\mathbb{Z}_2$-graded vector space, with $\ell\mathbb{H}=\mathbb{H}\ell$, and then $\mathbb{O}\otimes_{\mathbb{R}}\mathbb{O}=(\ell\mathbb{H}\oplus\mathbb{H})\otimes_{\mathbb{R}}(\mathbb{H}\oplus\mathbb{H}\ell)$ becomes $\mathbb{Z}_2\times\mathbb{Z}_2$-graded:
$$ (\ell\mathbb{H}\otimes\mathbb{H}) \,\oplus\, (\ell\mathbb{H}\otimes\mathbb{H}\ell) \,\oplus\, (\mathbb{H}\otimes\mathbb{H}) \,\oplus\, (\mathbb{H}\otimes\mathbb{H}\ell). $$
(All unadorned $\otimes$ are over $\mathbb{R}$.) Notice the subspace we're quotienting by is also $\mathbb{Z}_2\times\mathbb{Z}_2$-graded, so we can quotient in each of the four "components" separately. Notice $\ell x\otimes y=(\ell x)y\otimes1=\ell(yx)\otimes1$ but also $\ell x\otimes y=\ell\otimes xy=\ell(xy)\otimes 1$, therefore the difference $\ell z\otimes 1=0$ vanishes, where $z=xy-yx=$ $2\,\mathrm{Im}(x)\times\mathrm{Im}(y)$ can be any pure imaginary octonion. The same cancellation occurs with $\ell$ on the right of $\otimes$, but with $\ell$ on both or neither side no cancellation occurs (but quaternion scalars can be slid to once side).
Thus the four $\mathbb{Z}_2\times\mathbb{Z}_2$-components become, in the quotient,
$$ \mathbb{O}\otimes_{\mathbb{H}}\mathbb{O} = \mathbb{R}(\ell\otimes1) \,\oplus\, (\ell\otimes\mathbb{H}\ell) \,\oplus\, (1\otimes\mathbb{H}) \,\oplus\, \mathbb{R}(1\otimes\ell). $$
So it would appear this "$\mathbb{O}\otimes_{\mathbb{H}}\mathbb{O}$" is a $10$-dimensional $\mathbb{R}$-vector space. There is a natural map $\mathbb{O}\otimes_{\mathbb{R}}\mathbb{O}\to\mathbb{O}\otimes_{\mathbb{H}}\mathbb{O}$ given by $a\otimes b\mapsto a\otimes b$, but the algebra's multiplication map $\mathbb{O}\otimes_{\mathbb{R}}\mathbb{O}\to\mathbb{O}$ does not factor through it like it would in the associative world.