The Earth is not a single rigid body, but consists of at least five separate regions which can move relative to one another. These are the crust (which is the region that we use to measure day length), the mantle, the core, the oceans and the atmosphere. Although the total angular momentum of the Earth may not change, these regions can and do exchange angular momentum between themselves over timescales ranging from days to decades. This leads to fluctuations in the angular velocity of the crust, and hence fluctuations in the length of a day.
This Wikipedia article describes some of the mechanisms by which the different regions exchange angular momentum.
Over long periods of time, the Earth and the Moon exchange angular momentum through tidal effects, leading to a gradual but steady increase in the average length of a day. This effect is of the order of a few milliseconds per century.