I would be swishing with **warm** salt water as often as I could stand it.
Warm was the edit...
You could also use an ice compress on it.
Used tea bags... cloves.
If you have cloves, add a bit of ginger (to help with taste) and make a tea, let it cool and swish it too.
it will help dull the pain.
If it is an abscess... and you know it, you could relive pressure by popping it with a sterile needle... but get ready for the pain of doing that :-O As long as the puncture hole is open, try to get as much gunk out of it as you can. If it starts to grow back... repeat - not fun.
keep up the salt water rinses and gargles.