To date, there is no evidence of a link between common pets and infertility. Having dogs, cats, hamsters or others will not affect your fertility. However, there exist certain parasitic infections that can be passed from them to humans and might have a negative impact on human fertility. As for cat allergy, there exist no studies that show a link between it and human infertility either.
On the other hand, people who work with livestock could be infected with certain diseases such as brucellosis, which symptoms include fever, chills, aches, sweating... In short, it might cause the body not to work optimally, and therefore affect a couple's reproductive efforts.