The correct and grammatical way to say the sentence would be "In the background, there is a white wall." If you said "In the background is a white wall," it would not make too much sense. You could say, however, "The background is a white wall." Because you're connecting "white wall" and background" with a linking verb.
If you said "In the background is a white wall," it is less of a formal way to say the sentence and is more of a style of writing. Depending on the type of writing you're doing, it can be suitable. It seems like it fits more into a creative writing type of style rather than a formal one.