I'm certain that the question is not correctly asked. Notice that in the sentence,
How can I calculate this sum?
1/1-(x+x^2)^2
there is no sum at all!
However, it seems that the OP wants to know if the expresion 11−(x+x2)2=∞∑n=0(x+x2)2n, is valid.
If that is the question... I'd say yes, as long as (x+x2)2<1.