Those of us who consider ourselves skeptics and supporters of science, and most especially those of us who are involved at some level in defending good science from the efforts of creationists to water down (or even eliminate) the teaching of evolution, will be familiar with this question. I think the answer is not simple and is much thornier, both philosophically and practically speaking, than many people (including many skeptics) would like to admit.
Let me first take a few minutes to outline some basics of the philosophy of science that are relevant to this discussion. This has to do with the nature of naturalism in science; more specifically, we need to make a very clear distinction between methodological naturalism and philosophical naturalism.
Methodological naturalism is the practice of naturalism in science; in other words, as it is most commonly stated, there are naturalistic answers sought for scientific questions, and the question of potential supernatural answers (“miracles” if you will) is not even considered. It was the application of methodological naturalism in what was in the 19th-century still referred to as natural philosophy, which helped to define and distinguish modern science as it is currently practiced. In the view of many scientists, science as practiced doesn’t necessarily speak to the validity or non-validity of the supernatural precisely because it is constrained to seeking only natural causes for the phenomena we observe in the universe. In the view of pure methodological naturalism, science is agnostic on such matters, and this gives many believers in the supernatural an “out” for accepting science while retaining their beliefs.
By contrast, philosophical naturalism is usually defined as a philosophical position that there is no such thing as the so-called “supernatural” because the natural world is all that exists. This view assumes, a priori, that there is no separate realm of existence, which is distinguished from the natural world. Thus, in this view, anything, which is claimed to exist within the “supernatural” realm, either doesn’t exist at all or is being confused for some other kind of natural phenomenon which isn’t necessarily well understood by the claimant. It should come as no surprise that in the world of the philosophical naturalist there is no such thing as a miracle and there are no gods per se. There is no comfort for the supernaturalists in the worldview of philosophical naturalism.
Having laid that foundation, let us now get back to the specific case of the entire evolution-creationism discussion, where we can see this distinction between the methodological and philosophical view of naturalism on display. There are many pro-science groups, such as the National Center for Science Education, which take the view usually credited to the late Stephen J. Gould called non-overlapping magisteria (NOMA) when discussing the thorny issues of science, religion, their intersection, and their conflicts. Basically NOMA takes a kind of modified position of methodological naturalism and is described by Gould as follows: "the magisterium of science covers the empirical realm: what the Universe is made of (fact) and why does it work in this way (theory). The magisterium of religion extends over questions of ultimate meaning and moral value. These two magisteria do not overlap, nor do they encompass all inquiry (consider, for example, the magisterium of art and the meaning of beauty)." 
Even the National Academy of Sciences in the United States takes a viewpoint based upon NOMA, wherein, in regards to the evolution-creationism issue, they state: "Scientists, like many others, are touched with awe at the order and complexity of nature. Indeed, many scientists are deeply religious. But science and religion occupy two separate realms of human experience. Demanding that they be combined detracts from the glory of each." 
Note that in the cases of taking the NOMA stance, there is nothing said one way or the other regarding the existence or non-existence of gods, miracles, or any kind of supernatural phenomena. However, there are many for whom the position of NOMA is rather unappealing, most notably because it seems to have the effect of stacking the deck in favor of what are considered unfounded beliefs and claims. For example, while the Catholic Church can tell its followers that the science for evolution is ironclad and therefore acceptable, that same religious institution routinely turns its back on science and completely ignores it regarding questions related to the authenticity of supposed religious relics such as the Shroud of Turin (which is, in case you didn’t know, a fake). This is merely one example where the believers and purveyors of the supernatural will try to have their cake and eat it too, the critics of NOMA would say, as they with one hand embrace science while with the other hand reject it.
Such inconsistencies and intellectual laziness have spurred a number of critics of NOMA to speak up in recent years, among them Richard Dawkins, Susan Jacoby, Sam Harris, and Victor Stenger. I would like to focus specifically upon Victor Stenger, a particle physicist and philosopher of science, because I like the manner in which he approaches a variation of this question in his book God, the Failed Hypothesis.
Stenger starts off by carefully defining “God” as the monotheistic god of fundamentalist Christianity (and Islam and Judaism) which many creationists invoke as the creator of the universe (no evolution required, they claim). Stenger then notes that the very believers in this God themselves often make a number of claims about his supposed powers that manifest themselves in the natural world around us. In effect, they are making empirical, naturalistic claims for something which is supposed to be supernatural and therefore beyond nature; and it is this blurring of the line between the natural vs. supernatural by believers which is one origin of the “having their cake and eating it, too” criticism. Stenger contends, and I agree with him, that if these believers are going to throw around empirical claims, then they have clearly thrown down the gauntlet and those claims can and should be tested empirically (i.e. using science). He then goes on in God, the Failed Hypothesis to show that modern science shows the monotheistic god of the fundamentalist Abrahamic religions doesn’t exist (though he holds out the possibility of a deistic creator god).
Going beyond Stenger’s work, let us just take a look at the decades of work performed by many prominent skeptics. For example, James Randi has made a career out of testing supernatural claims, most famously through his Million Dollar Challenge. In addition, just look at any number of supposed “miracles” that have been clearly shown to be anything but by skeptical investigators such as Joe Nickell and Ben Radford over the decades. I would think that all of these skeptics, and many more besides, would agree that if the believers are going to claim some sort of empirical basis for their belief in the supernatural, then those claims are fair game for scientific analysis and testing.
So, where do I stand on all of this? As I’ve probably implied, I appreciate and agree with much of the stance adopted by Stenger, Randi, Nickell, and many others who are more than eager to apply scientific scrutiny to those empirical claims of the supernatural. And I say that precisely because of those believers who want to try to abuse NOMA for their own ends: they wish to claim empirical science for their purposes when they think it works for their belief system, yet they spurn (and even attack!) empirical science when they think it challenges that same system. This is, as I like to call it, “heads I win, tails you lose” argumentation.
However, that said, I am not willing to completely abandon NOMA, because while I do not believe in any gods or other aspects of the supernatural, I cannot definitively say it does not exist (though I’m willing to bet my non-existent soul that it doesn’t). For NOMA, what I will say is this: I am willing to allow the believers their illusions, so long as they don’t make empirically-based claims as to their validity here in the real world. Once those claims are made: game on!
Matt Lowry is a high school & college physics professor with a strong interest in promoting science education, skepticism and critical thinking among his students and the population in general. Towards these ends, he works with the JREF on their educational advisory board, and he also works with a number of grassroots skeptical, pro-science groups. In what little spare time he has, he blogs on these and related subjects at The Skeptical Teacher.