When we have a set, is it correct to refer to another set as 'itself' when this other set is merely equal to it?
More formally, I am asking whether or not two sets with the same elements can be considered to be two separate mathematical objects.
I am a programmer so I am used to the "same object" being a distinct notion to "the object to which this object is equal". Therefore, this has led me to consider whether or not this concept is true within mathematics.
To understand the foundational axioms upon which mathematics has been built, you will need to study a course on Set Theory. The axiom that answers your question is known as Extensionality:
Axiom of Extensionality: Sets are uniquely determined by their elements.
This means that two sets are equal if and only if they have the exact same elements. The order of the elements of sets does not matter in mathematics.
If the sets $A,B$ are equal, then $A$ are $B$ are just two names for the same thing. These are not and cannot be distinct mathematical objects, otherwise this would violate the Axiom of Extensionality.
The above is the short answer to your question. Here is the slightly deeper answer.
If you want to know why this is true, then things get a bit philosophical. This is an Axiom - which means that it is something that mathematicians say is true. You don't necessarily have to accept the axioms and many mathematicians work with different axioms (for example, you may have heard that the Axiom of Choice is somewhat controversial). The underlying assumption here is that we are working in $ZFC$ Set Theory (the most common axiomatic system used by mathematicians today). The Axiom of Extensionality is one the axioms of $ZFC$, but it's perfectly reasonable to explore other systems of axioms which may not necessarily accept Extensionality.
For example, you may be interested in this MathOverflow post which discusses some of the research that has been done on Set Theory without Extensionality.
To wrap this answer up, before things get too far into philosophy and away from mathematics, the axioms of set theory essentially tell us what sets are and what sets aren't. There isn't a clear, precise definition of a set. They are just objects governed by the axioms of set theory. Therefore, we can change the axioms (which will change what constitutes a set) to suit what we are doing. Although we do need to be careful, because if you change the axioms then sometimes things that are true in one system will be untrue in another. This doesn't mean that one is right and one is wrong, it just means that you need to be weary of which results depend on which axioms.
There is an important sense in which this is “the wrong question”. Set theory is completely agnostic on the question of whether two equal sets are “the same object”. They could be, or not, but either way, set theory does not care. The whole point of set theory is that it is the theory you get when you ignore all such distinctions.
Set theory was invented precisely to study the behavior what you get when you have objects called ‘sets’ with ‘elements’, where ‘sets’ are considered to be indistinguishable whenever they have the same elements. This is the real content of the axiom of extension. Whether the two sets are “the same” or not in some philosophical sense is not a set-theoretic question. The meaning of what it means to be an element of something is also not a set-theoretic question. Are clouds elements of the sky? Set theory has no opinion on this matter.
FD_bfa says in a comment “[Extensionality] is an Axiom - which means that it is something that mathematicians assume is true.” I think this is philosophically incorrect. If there were real sets in the physical universe (the way there are real computers with real memories) we could adopt an axiom, assume it was true, and maybe discard the axiom if it conflicted with experiments on real sets. Then we would be doing physics, not mathematics. I think my paragraph above is a more accurate description of the situation. The axioms do not describe what we believe sets are like, because there are no actual sets, and they are not ‘like’ anything. Instead, axioms describe the properties that we want sets to have, the properties that we consider interesting in a particular context. The axiom of extension is saying that by ‘sets’ we mean certain abstract objects whose properties are completely determined by their elements. There might be things like two sets that have the same elements but are different, but we are not, for the time being, interested in distinguishing them.
For a more familiar example of the same sort, consider the number $3$. Three apples is not like three pencils, because the first one is apples and the second is pencils. But the the whole point of the theory of arithmetic is to ignore the differences between three apples and three pencils. Apples and pencils are not arithmetic objects. The number $3$ is what is left when you take three apples and ignore the apples. In arithmetic, are all threes the same object, or could there be different threes that are numerically equal? This is not something that arithmetic has an opinion about. Arithmetic is only concerned with numerical equality, not with some other notion of identity.
Mathematical objects are abstract in a way that computer objects are not. Two computer objects are considered different if they are stored in different parts of the computer's memory, even if they are considered to have equal values for one reason or another. This has real-life implications for the engineering of computers and computer programs. There is nothing analogous in mathematics, which is not engineering or physics.
