find coefficient of generating functions with three variable

savecancel

Well... your expression is a rational fraction of two polynomials, of degree 6 in $x$. There is no such thing as an $x^{15}\dots$ coefficient.

However, this fraction can be approximated (Taylor development) as series in $x$, whose coefficient in $x^{15}$ may contain terms in $d^7$, which may contain a term in $r8$. An obvious way to search is :

sage: foo=1/(1-((x*d+(x*d)^2+(x*d)^3)/(1+x*d+(x*d)^2+(x*d)^3)+(x*r+(x*r)^2+(x*r)^3)/(1+x*r+(x*r)^2+(x*r)^3)))
sage: foo.series(x, 16).coefficient(x^15).expand().coefficient(d^7).expand().coefficient(r^8)
3296

I use free version of ChatGPT,

Don't. For mathematical purposes, ChatGPT (and other LLM engines) are (expensive) noise|bullshit generators. Do it yourself.

EDIT : lazier alternative:

sage: R1.<x,d,r>=LazyPowerSeriesRing(QQbar)
sage: foo=1/(1-((x*d+(x*d)^2+(x*d)^3)/(1+x*d+(x*d)^2+(x*d)^3)+(x*r+(x*r)^2+(x*r)
....: ^3)/(1+x*r+(x*r)^2+(x*r)^3)))

foo is the (infinite) polynomial approximating youir fraction with an arbitrary precision. The term you seek is of total degree 30, therefore one of the terms of :

sage: foo[30] # Alternative : foo.coefficient(30)
x^15*d^12*r^3 + 65*x^15*d^11*r^4 + 546*x^15*d^10*r^5 + 1860*x^15*d^9*r^6 + 3296*x^15*d^8*r^7 + 3296*x^15*d^7*r^8 + 1860*x^15*d^6*r^9 + 546*x^15*d^5*r^10 + 65*x^15*d^4*r^11 + x^15*d^3*r^12

eliminating the terms of too high degree in $d$ and $r$ gives us the relevant term :

sage: foo[30].truncate(d, 8).truncate(r, 9)
3296*x^15*d^7*r^8

HTH,