You ask for a sprawling city with 4 foot tall structures (external), without the ability to dig to increase the internal height.
Whether it is plausible
As some have indicated in comments and other answers, people tend to live inside their homes, and as such they are constructed to fit people, and typically people will try to improve things past basic utility into comfort. In any culture you have a number of contributing factors such as security, tradition, and economics. We need to tweak these factors to produce an environment conducive to the maintain the status quo of tiny buildings.
But before we start in on those factors I'd like to start with two points:
- If you only retreat to your personal space to sleep, you do not need a lot of head room.
- The primary requirement for any shelter is not for comfort, but for protection from the elements. Small buildings will satisfy this basic need as well as (or in some environments even better than) taller ones.
Today, we have a sprawling city of four foot tall buildings, where the average adult can see very long distance in all directions. Most who live here today do not question it, for it has always been this way, but how did we get here to begin with?
Founding
Stepping back many many generations before any trade routes run through the area. We find a vast and barren wasteland of wind scoured granite fields resulting from a catastrophic natural (or magical if your world allows it) disaster. At the center of the stone fields we find some grand awe inspiring natural landmark formed as a result of the same event.
The first settlers in the area are ascetic monks come to worship at the landmark. They build their shelters with the minimal and poor quality natural resources available locally. Although the tiny hovels they can manage push the limits of their resources material strength (built any taller or any wider, the structures collapse under their own weight) they fit perfectly with the monks' lifestyles of self denial providing only shelter from the sun and wind.
Time passes and the environment begins to recover from the disaster, enabling it to support a larger population. Along with this recovery, the community grows, and as it does it becomes more formalized. Higher quality materials are retrieved to build more durable replacements for the original structures as well as additional ones for new arrivals; however, the mold has been struck for what is expected of the supplicants, and the replacements are no larger than the originals.
Generations Pass
More generations pass and the site draws the attention of others. The now well organized Order of the Landmark grows rich on the gifts brought by pilgrims which travel great distances to visit the now Holy Landmark.
Given the benefit of time and the care (and prosperity) of the Monks of The Order the area immediately surrounding The Landmark has recovered as much as can be expected from the disaster has become quite a pleasant place to live. With the exception of the middle of the day when the sun is highest in the sky, the climate is perfect to support a culture where people live the majority of their lives outdoors.
The riches and the pilgrims attract merchants and other residents to support the local tourist industry but The Order is still in control and issues an edict that "No permanent structure may obstruct the view of the Holy Landmark".
Although not an explicit height restriction, the practical result of the edict limits the height of buildings to below eye level of the average high ranking monk.
Minima Civitatis
As the city is the center of travel for the faithful from all directions, it naturally becomes a major center of commerce as well and The Order further enriches itself on taxes from trade.
Centuries of tradition enforced by the city's primary resident results in a society where most 'private residences' are limited to smaller sleeping quarters with most day to day activity taking place outdoors. Kitchens and areas where work is done will most likely be completely open air, or built to allow pop-up tent like roofs.
So why not down?
The entire settlement is built pretty much directly on bedrock. Digging into the stone is possible, and it will happen, but for most people it is not economical to do so to gain a little extra head room you don't need anyway.
That said, some digging will be required. Gutters and sewers become necessary to carry away human waste with settlements of any reasonable size to avoid disease.
Perhaps the city is littered with public outhouses (or trenches anyway), but these still need to lead somewhere.
Beyond public utility work like the sewers digging where it does happen will be for the homes of the wealthy, or for shared indoor community space where the benefit of the effort is shared by many.
Center of Trade?
As with the need for sewers, here your restriction against any digging becomes a problem which is likely to prevent it from becoming a center of trade.
Trade means product. Which means storage. More trade means more product and more storage. Which means warehouses. No sensible person is going to want to deal with warehouses with a 3 to 3 &1/2 foot interior. It would be impossible to store or retrieve anything efficiently. Trade is one of those areas where economies of scale really impacts things, and the height restrictions literally prevent any scale here.
If the height restriction stands, then warehouses will have to be one of the exceptions to the no digging rule for the city to become a trade center. Otherwise the warehouses will be built outside the city, along with the offices and businesses that handle the trade as well as the homes of the merchants themselves, and the people that work for them etc, etc. All forming perhaps a sister city to yours, but that city would become the center of trade and reap the direct benefits and riches associated with it.