No, not default as in default on debt—default as in “what Greece can do if there is no agreement with the EC/ECB/IMF”. This is a game theory term. In bargaining theory, everything depends on the cost to parties of not agreeing: the weaker your default option, the worse your expected bargaining outcome.
Broadly speaking, Greece is vulnerable on two counts. First, it relies on loans from the Troika institutions to service its loans to them. This is essentially a shell game, and Varoufakis is right that pretending it solves anything is not the solution but the problem. Nevertheless, in practice this means that if further loan tranches are not delivered and debt payments aren't made, Greece defaults—using default now in its finance sense. With its primary surplus, funding the Greek state is not the issue; rather, default exposes Greece to political retaliation, expulsion from the eurozone or from the EU altogether. There have been noises from various quarters that unilateral refusal of debt service is a fallback option for Greece, and of course it is a theoretical possibility, but I doubt anyone really wants to see where it will lead. Nevertheless, since default in the finance sense is so hazardous for Greece, it makes a lousy default in the game theory sense. Bad for their bargaining position. Greece needs some other form of leverage on this front.
The second vulnerability is in their banking system. This in turn results from two factors. The first is that Greek banks have been repositories for the funds of domestic oligarchs, and Syriza has promised to go after this crowd. That means capital flight, which is not a currency problem (because of the euro) but is a financial stability problem. Assuming that Syriza is serious about political and economic reform, capital outflows are unavoidable. (It would be nice if the rest of Europe cooperated by bringing transparency and accountability to flight capital, but why start now?) Thus shoring up the financial system is a concomitant of attacking the cronyism that has bled the country for decades.
Meanwhile, Greece is vulnerable to old-fashioned garden variety bank runs. There is no clear lender of last resort for Greek banks or for any other banks in the eurozone for that matter. National governments can’t open the spigot unless they've already got the fiscal space for the job, and Greece obviously doesn't have it. This is because, in theory, they can’t print euros. Meanwhile, the ECB's formal mandate does not include LOLR, nor have they made any credible commitment to perform this function; it is strictly ad hoc. If there is a run on Greek banks, who protects depositors?
I will say again that, in this respect, Greece is not helpless. By far, the preferred option is an agreement that invokes ECB backstopping, but what if there is no agreement? What’s the default option? I think Greek officials should be taking a close look at the possibility of printing euros for the sole purpose of redeeming insured deposits. On one level this is a flagrant violation of EU rules. But, while a violation of the letter, it is not hostile to the spirit: wiping out deposits eliminates money, and redeeming them simply restores the same amount of money. It just replaces deposit-money with paper-money. Would there be screaming and yelling from Berlin and its accomplices? Yes, of course. Would there be consequences for Greece? Unclear. I’m not saying this is where Greece wants to go—it isn't—but it constitutes a potential default option if bargaining breaks down and a bank run ensues. What’s the alternative?
Just to be clear: default options are not what you want to do. They are what you have to do if there’s no agreement, and, in a bargaining-equilibrium world (which Varoufakis argued against in Rational Conflict) their main effect is in altering the terms the parties finally arrive at.