I think it may taste bad because it instantly starts to kill pathogens, germs, bacteria etc. They start bubbling up and THAT makes for a nasty taste!

All answers were correct, I think

Pathogens, bacteria are dying off
H2O2 is made with metals, various processes can be used - thats why you are supposed to use FOOD GRADE , and at 35% diluted with distilled water, there isn't as much, and its not harmfull, and it won't taste as strongly that way.

In fact, I don't think my mix tastes at all.
I usually use some baking soda to brush with after a minute of H2O2 (Hydrogen-Peroxid) 3% diluent, and the baking soda is what I taste then of course.

I think its ok to use them together, so use the "bs" first.

But I really think my mix doesn't taste~! certainly not like the 3% mix in the brown plastic bottles in the drug store - yecchhh!!!!

*****
My mix is Food Grade 35% with 11 parts distilled water = 3% [pretty close].

I use the cheaper "non food grade 35%" for laundry and cleaning and killing the stubborn mold in my toilet. I have a liter of each type food grade and regular.

*****