I have heard this argument against agnosticism:

"If you told an agnostic that you had an invisible third arm, they wouldn't believe you even though they can't disprove it."

This attacks the premise that agnostics don't rule out any possibilities until they are proven or disproven. However, there are degrees of probability (all-but-certain to extremely unlikely) and I believe that the existence of a god is more likely than someone's invisible third arm.

Any relevant responses are welcome, I'm curious what other perspectives you may have.