No, you don't have to have many stones in the gallbladder to have pain. Some people have pain from just a few stones irritating the GB wall or while trying to escape.
I have heard of several people having their stones analysed, but none included the presence of olive oil. Some cited presence of organic matter, noxious substances, metals, bacteria, and of course, typical calcified material as found in gallbladder stones. To turn golden color olive oil into bright green pea-size stones or bigger would a miraculous feat by the body, but then some also believe that water can be turned into wine. To "manufacture" 80 grams of stones from a mere 1/2 cup of olive oil, would even be a greater miracle. And some people just need to have doubts to feel self-assured, despite feeling so much better because of cleansing. Or perhaps, is it because of the Placebo?