Exactly.

Feedback gets even the tinniest amount of noise and, by mixing it again with the original signal, the amplitude is increased until we get clipping.

Methinks that if we want to stick with this idea, we have to put some kind of threshold to prevent the noise from sneaking in. What do you think?

There is also another way of doing this, by using the stereo mixer which provides for amplitude modulation of the input signals. It's like having your hands on the sliders of a DJ mixer and boosting/cutting left/right sides, L(up), R(down), then L(down), R(up) and so on.