1999 mercury cougar won't start.?

I have a 1999 mercury cougar. 2.5 liter v6 I've been having trouble recently getting it started. The problem is it wont turn over and has been getting worse. When it does turn over it starts normally, but when it won't start, it doesn't do anything. The normal lights come on but it doesn't click like a dead battery, and the fuel pump doesn't turn on for a second or two like it is so post to. I have checked the battery connections, alternator, and battery. All are good. I would think it was the starter but it doesn't make sense that the fuel pump doesn't kick on when It doesn't start. Usually it will start if I let it sit a day or two but then it does it again a few starts later. Is there a main fuse that controls the starter and the fuel pump? If not any ideas on the problem? Any ideas are welcome!