I think symmetry was well addressed in the other answers. But since there was no mention of Lie algebras or infinitesimal generators, I'd like to add this answer, since these notions appear often in physics.
Many of the "ordinary" groups that we encounter in a first class on group theory belong to a class of groups called "finitely presentable groups". For such groups you can choose certain elements of that group as generators, and describe certain relations in terms of the group operation on those elements. It turns out that these generators and relations describe the group completely.
So for example $\mathbb Z$ is generated by $1$ (with no relations): $\mathbb Z = <1>$, and $\mathbb Z_5$ is generated by $1$ subject to the relation $5*1 = 0$, or: $\mathbb Z_5 = <1 | 5*1 = 0>$. In a way these generators and relations capture everything there is about the group. The generators act as "small building blocks" for the group. But there are groups for which this type of analysis doesn't work.
Lie groups are another class of groups that are also "manifolds" - i.e. they look like curves or surfaces in n-dimensional space. In such groups the "generators" are not elements of the group - instead they are infinitesimal objects (vectors!) that belong to a different space (a vector space! The "tangent space at the identity"), and the relations between them are defined not in terms of the group operation, but in terms of a new operation (the Lie bracket).
In more detail:
You need these vectors as generators because in a smooth group (think $\mathbb R$ as opposed to $\mathbb Z$, or the circle $S^1$ as opposed to $\mathbb Z_5$) you cannot choose a "smallest" element as a building block (there is no "smallest" real number in $\mathbb R$). So instead you need to get fancy and take limits of sequences of operations as they approach the group's identity. Taking these limits requires doing calculus on the surface, which is what you do in differential geometry. That's why this type of analysis is done on smooth groups (i.e. they are differentiable manifolds), and why Lie groups are required to be smooth. Because these generating vectors are obtained from a calculus-y limiting process, they are called "infinitesimal generators".
The way these vectors "generate" the group is that they sit with their base at the group's identity element and "point" in the direction of the operations whose limits they came from. You can then "start at the identity and follow a vector" in order to generate those operations. You can also "mix" vectors together using a special operation, by saying something like "let's follow $v$ and then $w$ and see what happens". This operation on two vectors gives you another vector (which is not obvious) - it's called the Lie bracket, written $[v, w]$. The Lie bracket of two "infinitesimal generators" (i.e. vectors) captures information that is similar to the relations between group elements in finitely presentable groups.
So now we have:
- finitely presentable groups: some "smallest" elements of the group can be used to build up the entire group, subject to some relations between those elements
- Lie groups: can't choose smallest elements, but we use limits and differential geometry to squeeze infinitesimal generators (i.e. vectors) out of our group. These vectors can "generate" the entire group[1], subject to relations between the infinitesimal generators via the Lie bracket.
To summarize: a Lie group is a smooth group that looks like a curve or a surface (possibly in n dimensions). It has "infinitesimal generators" that are tangent vectors sitting at the group's identity. We can "mix" these tangent vectors to get more tangent vectors using the Lie bracket. This vector space plus the Lie bracket is a Lie algebra. All together we get this calculus-y version of generators and relations.
Some final notes on notation that comes from Lie theory / differential geometry and that often looks like black magic:
tangent vectors in differential geometry can be presented as "derivations", so you might write a tangent vector as $\frac{d}{dy}$. This can be mind-blowing when you first see it. When this happens in the context of Lie groups, that tangent vector $\frac{d}{dy}$ is often called an "infinitesimal generator of Y", for whatever Y represents. For example $\frac{d}{dx}$ might be an "infinitesimal generator of translations" (in the X direction), because the group this comes from represents translational symmetry.
Since the Lie bracket is written $[v, w]$ for tangent-vectors / infinitesimal-generators $v, w$, and such vectors in differential geometry are often written as derivations e.g. $\frac{d}{dx}$, you'll often see expressions like $[\frac{d}{dx}, \frac{d}{dy}]$ (I'm looking at you quantum physics). Such notation is often an indication that there are Lie groups lurking in the background.
the function that lets you "start in the direction of a tangent vector and follow the line of operations it points to" is often called... $e$. So for example you might write $e^{t\frac{d}{dx}}$. This is the "one parameter subgroup" generated by $\frac{d}{dx}$. If you see weird things[2] being raised in an exponential: matrices, differential operators, etc... another clue that Lie groups are lurking about.
[1] subject to certain conditions of course
[2] if $e^{i\theta} = cos(\theta) + isin(\theta)$ ever bothered you, it has a beautiful interpretation in terms of Lie groups