noisy babies/toddlers in public

I have often had to make the sacrifice to leave a public place early (e.g., a cafe or grocery store) because parents bring their babies, or sometimes young children, and they start screaming. And sometimes it can last for a while. And many people seem to think it's okay, since they are babies, and babies cry. But on the other hand, the parents can be more responsible and leave the public place. Some parents are more considerate and try to not stay in the same place forever when they have a screaming baby. While others seem perfectly fine to continue having their tea in the cafe or spend their time shopping while their children are being noisy. I find the screaming really hard to bear, and it's even physically painful sometimes and that I want to cry. I often leave if it continues for too long. But it seems unfair that it's me leaving instead of the baby making the noise in the middle of 20-30 people leaving. Any ideas on how to cope when a people bring noisy babies/toddlers? Or anyone else feel that it is unfair?