Satan is so crafty and this is why it is important to know a person's position and how they define things before arguing so you are not caught debating straw-men. The first thing I would ask is "What do you mean by atheist?". There is really a blurring of two different views here. The first (traditional atheism) is when a person is making a truth claim that God does not exist. The second way to take the term 'atheist' is a person who lacks a belief in God because they haven't investigated the evidence. With the first person they must absolutely defend their poisition for they are making a truth claim along with the Christian. We say God exist they say He does not. We both share the burden of proof. With the second position though they want to have their cake and eat it too. They are technically saying "I don't know if God exist and therefore I do not have any burden of proof". That is fine if they wanted to stop there but that's the problem is that they can't. The further go on to argue (illogically) that "Because I don't know if God exists then therefore He doesn't exist". Now they have driven themselves right back into the burden of proof problem that the traditional atheist has.
Furthermore notice the non sequitor here. Just because you don't know if something exists doesn't mean it doesn't. Lets take ourselves back a few hundred years to the debate on whether the earth was flat or round. Let's say an inquisite person hadn't come to a firm conclusion yet and stated "I don't know if the world is round." Would that make the world flat?!?! Of course not. It is more a statement about their worldview (pardon the pun) rather than a truth claim. Now if they went on to say "I don't know if the world is round therefore it isn't round." then we would call them ridiculous. This is the same sort of semantical gymnastics the neo-atheists are trying to play on us today. Beware of their craftiness and don't get caught in the world games. Make them define their poisition. For someone to hold a position that they don't have to defend they must state "I don't know if God exist" and stops there. But if they do that they cannot reject the Christian position. For as soon as they do (by saying "God does not exist") then they have made a truth claim and must support it.