Start with an isosceles triangle with a vertex angle $\alpha>\frac{\pi}{2}$ and draw two similar triangles on its sides, continue ad infinum. Each triangle will have sides $\frac{1}{2 \sin \alpha/2}$ smaller, or the area $\frac{1}{4 \sin^2 \alpha/2}$ smaller. For $\alpha>\frac{\pi}{2}$ the final area of the fractal will be finite.
In the general case the boundary of the resulting figure will have a fractal shape. However, I'm interested in the borderline case of:
$$\alpha=\frac{2 \pi}{3}$$
Then for the first 2 steps we obtain:
And in fact every "groove" like the central one will be filled completely by the smaller triangles, because we have:
$$S_n=\frac{1}{3} S_{n-1}$$
As $2$ new triangles are created at each step, we have:
$$S_0 \sum_{n=1}^\infty \frac{2^n}{3^n}=2S_0$$
One half of this area ($=S_0$) will be spent to fill the groove, the other will stay "outside". (This is wrong see the paragrath in the end. Actually the everything goes inside the groove, since half of the triangles overlap the older ones).
As I ask in the title: what is the final shape of the boundary of this fractal going to be? Will it have a smooth $1D$ boundary or will it stay fractal?
Unfortunately, I haven't been able to just program this fractal yet, but also I would like to hear a theoretical answer if it's possible to determine without doing the experiment.
I have started to figure out how to build the program in cartesian coordinates. Basically we have a one dimensional array of points (collected over all previous iterations) and between each neighbour points we build a perpendicular with the length $l_n=l_{n-1}/ \sqrt{3}$. Then we add this new point to our array exactly between the two points, and continue further down the array.
If two initial points have the coordinates $(x_1,y_1)$ and $(x_2,y_2)$ then the new point will be:
$$x=\frac{x_1+x_2}{2} \pm \frac{y_1-y_2}{2 \sqrt{3}}$$
$$y=\frac{y_1+y_2}{2} \pm \frac{x_1-x_2}{2 \sqrt{3}}$$
The choice of signs is not always easy, I have not determined the universal rule yet. Here's the result of 4 full iterations and the start of the 5th, made by hand:
An important correction based on the comment below: the figure eventually starts to overlap itself. But because the triangles add up perfectly to fill the "grooves", let's just add a rule to stop constructing new triangles in places the old ones touch each other with their sides.
I used LibreOffice Draw to get more iterations in a crude way (by copying and scaling the whole previous pattern) and I got this at just the 3rd step. Looks like a fractal boundary after all (though the impression might be wrong, since more iterations should cover all the "grooves").
Because of the overlap reveal I have also added a second question: what is the area of the fractal if we don't count the triangles which overlap the previous ones?