In principle, there is no association between Mirena being removed and female infertility. Many women have reported becoming pregnant almost immediately after the removal of Mirena.
In fact, the recovery rate for fertility levels seems to be higher than with other contraceptives such as Depo-Provera. On the other hand, it is true that it might increase the chances of miscarriage. To sum up, the answer is no, it does not cause infertility, but the chances for complications to appear are higher.