I am currently in the same position. I had two silver fillings removed by a regular dentist about two months ago. So far they are still in the healing process and so far nothing major that has worried me intensely.
I went ahead with the regular dentist route because my mother had all of her silver fillings removed with that same regular dentist 3 years ago and her teeth look great and shes had no problems.
However, it was brought to my attention that silver fillings are dangerous to remove and if it is not to necessary to never remove them. So yeah, it is a controversial subject. I will not be removing anymore unless it is with a biological/holistic dentist.
I was also told I have 3 wisdom teeth that need removal, and I'm now trying to figure out which is more important to remove silver filling or wisdom teeth. For tooth extraction you do need an oral surgeon. There's really no way around that.