simple digital filter implementation with weird result

hi, I tried to implement the following digital filter : 1/8^3*[1 - z^-8]^3 / [1 - z^-1]^3. In order to do this with the filter function, I used the following code:

B_tmp = [1,zeros(1,7),-1];
B = conv(B_tmp,conv(B_tmp,B_tmp))/8^3;
A = conv([1,-1],conv([1,-1],[1,-1]));

I tested this filter on a slowly varying cosinus (9s period), with a sampling frequency of 3.2kHz:

X = [zeros(1,10000),0.5*(1-cos(2*pi*(0:(1/3200):(100*9))/9))];
Y = filter(B,A,X);

The result is weird : after a while, Y diverges from X:

figure;plot([X;Y]');

What did I do wrong here? Thanks