The Gibbs paradox deals with the fact that for an ideal gas with $N$ molecules in a volume $V$ seperated by a diaphragm into two subvolumes $V_1,V_2$ with $N_1,N_2$ particles in each subvolume, removing the diaphragm gives a nonzero change in entropy, but the change should be zero.
I don't understand why (conceptually) the change of entropy in this situation is supposed to be zero. Why isn't it positive - after all, removing the diaphragm gives the particles more freedom and thus increases the 'disorder' of the system and with entropy being a measure of this 'disorder' it should too increase. Conversely, if I put more and more diaphragms into the container, I could potentially isolate each particle in its own subvolume and leading the system to be very ordered, so the entropy should be very small.
What is wrong with this way of thinking?