Cantor defined the ordinals in his early work. Zermelo later proved that under the axiom of choice every set can be well ordered.
Well orders are very rigid in the sense that if $A\cong B$ are two well ordered sets then the isomorphism is unique. This allows us to construct explicit well orders for each order type.
Zermelo's ordinals were $\varnothing=\{\}$ for $0$, and $n+1=\{n\}$ for successors. The set of natural numbers is simply all the finite ordinals, however this could not have been defined in his original theory.
However in his set theoretic work, von Neumann popularized the axioms added by Fraenkel and Skolem (as well by himself) to Zermelo's early work on axiomatic set theory. He added the axiom of foundations and the schema of replacement. He then continued to define ordinals as transitive sets which are well ordered by $\in$. This work was perhaps popularized even further by Bernays and Goedel when they developed the extension of ZF which allows proper classes.
von Neumann's definition was that $0=\varnothing$ and $n+1=n\cup\{n\}$. This is a terribly convenient definition, since it allows us to say with clarify that $n< m\Leftrightarrow n\in m$, furthermore it allows us to define the natural numbers as the least inductive set.
von Neumann's idea easily carried over to the transfinite ordinals as well, I am not sure about Zermelo's convention. I do recall reading that Zermelo was less worried about ordinals, though.
(I also suggest the Introduction section of Akihiro Kanamori's The Higher Infinite in which he writes about the history of set theory. It might not be exactly about ordinals or the natural numbers, but it is an interesting read and it gives a few insights to add on what I have written above.)
Added: When searching more on the topic, I ran into Kanamori's recent paper cited below. There he says that Bernays worked with Zermelo, and it is seems that he was the one popularizing the Zermeloian definitions, as von Neumann's work relied on well foundedness of $\in$, and the replacement schema.
- Akihiro Kanamori, Bernays and Set Theory. The Bulletin of Symbolic Logic, Vol. 15, No. 1 (Mar., 2009), pp. 43-69