I have always drank coffee and it never interfered with my flushes. I also think the hot coffee/caffeine will speed peristalsis too and help expel some stones.
If you consider sludge, old bile, and chaff to be toxins. They come out in the flush also. Also, with colonics or coffee enemas done after flushing, you'll remove mucous and other toxins from your lower bowel as well.