Don't get me wrong, I'm still a virgin and will be until marriage. I just do not want to make my decisions based on feelings of implied morality, I want to take the path which actually benefits more people.
A monogamy pact (which is what a legal marriage today is) should not be broken. I just do not see why I must participate in this modern custom of promising to be monogamous when it is not actually necessary for morality.
The question is not of whether I will wait until marriage for sex, but whether I wish to bind myself permanently to a monogamous lifestyle. If I do not, but elect to marry in a manner that permits polygamy, it will of course present certain challenges that monogamy circumvents. But that doesn't make it wrong. Staying single circumvents plenty of problems, but I intend to marry nonetheless.
In a growing population where men marry later than women, there will naturally be more involuntarily unmarried women than men. It's more than a desire for multiple women that makes me consider this idea seriously, I honestly think it might be an improvement over the current condition.
Some rules for polygyny should still apply. A man with a wife/wives should not marry another woman without his current wife's/wives' consent. I imagine this would be the same as a complete monogamy pact in the majority of cases. Nobody should marry against their conscience.
If polyandry or mixed polygamy is allowed, it complicates things quite a bit. I'm not 100% sure this is wrong, but I can see why the idea would make people more uncomfortable.