I have been passing green stones all week without taking olive oil. That should answer your question.
I take chanca piedra which softens stones and I often pass them in BM's.
On my fourth flush I never took the olive oil and still passed well over 100 tiny stones. In the four other flushes I have done where I drank the olive oil I have passed in the neighborhood of 500 stones.