For most people, it seems that the symptoms are worst when the IUD is in your body, so try not to worry too much about symptoms you haven't had. You'll probably never have them! The consensus on this forum (and from what I've read elsewhere) is IUDs with progestin can confuse our bodies' hormonal balance. It takes time for it to right itself after the IUD is removed--usually three months or so, sometimes longer. But you only had it for six months--a pretty short amount of time compared to many others. Many women have a "crash" between four and six weeks after removal, as the artificial hormone works its way out of your system and before your body's own hormones have had time to re-balance themselves. But not all women experience that.
My own impulse would be to wait with the birth control pills until your body has had some time to right itself, and maybe consider non-hormonal forms of contraception. Some people are more sensitive than others to the artificial hormones. Also, keep in mind that anxiety is one of the most common side effects of the hormonal IUD. If you're feeling really anxious right now, it might be because of that--and it will get better!!