While Ingenuity would theoretically eventually hit a "service ceiling" above which the atmosphere is no longer dense enough to provided a TWR (Thrust-Weight-Ratio) of above 1, I suspect the major limit is Ingenuity's battery life/flight endurance.
According to the specifications from Wikipedia, the critical numbers are:
Maximum range, radio: 1000m
Flight time: Up to 167 seconds per flight
Maximum possible speed: Vertical: 3 m/s
So, disregarding acceleration and assuming that the flight profile is a straight ascent followed by a (controlled) descent, you would be able to budget 83.5 seconds to ascending which would reach a height of around 250 meters (double that if a crash-landing is acceptable).
To validate that atmospheric pressure reduction with altitude plays no significant role at this scale, the following formula can be used:
Barometric formula for pressure at height h:
$$P(h) = P_0 \times \left(1 - \frac{L \times h}{T_0}\right)^{\frac{g \times M}{R \times L}}$$
Variables:
- $P(h)$ is the pressure at height h
- $P0$ is the pressure at sea level (or surface level in the case of Mars)
- $L$ is the temperature lapse rate
- $h$is the height above sea level
- $T0$ is the temperature at sea level
- $g$ is the acceleration due to gravity
- $M$ is the molar mass of the planet's air
- $R$ is the universal gas constant
Constants for Mars:
- $P0$ = 640 Pa (Pressure at surface)
- $L$ = -0.0045 K/m (Temperature lapse rate)
- $T0$ = 210 K (Temperature at surface)
- $g$ = 3.72076 m/s^2 (Acceleration due to gravity)
- $M$ = 0.04334 kg/mol (Molar mass of Mars' air)
- $R$ = 8.314 J/(mol·K) (Universal gas constant)
Evaluating for our 250-meter flight, we get about a 2% drop in pressure, which is negligible.
As for software, while I don't know exactly what Ingenuity runs, I don't see a reason why it couldn't be modified to enable flight to much higher altitudes. Yes, it relies on optical flow (camera sees ground) to stabilize, but (as someone who has experience building and flying drones), there shouldn't be any reason why an IMU-only approach couldn't be used if it is so high that the camera can no longer properly see the ground (although it would probably not land in exactly the same spot as where it took off).