No caliber is too small, if it is the only thing available, or it is the only thing you can reasonably carry or hide. On your person or off your person. I do have and carry a North American Arms 5 shot mini-revolver, 1st choice NO, something I can put on a string and hang around my neck, YES! Put in a watch pocket of my jeans, YES. As the old saying go's, " Better than a sharp stick in the eye, but barely ". The man I bought it from, carried it in a vinyl eyeglass case, in his front shirt pocket. He glued the lower part of the case together and just left room for the little pistol. Pretty clever. 
+10 & Like the man said, the best gun is the one you will have on you.
Life demands tradeoffs: maybe you have to live in a less than ideal neighborhood for now, maybe you have business that takes you through some nasty areas, or maybe you live on 100 acres of mountain with a nice field of vision and ready made killzones and hardly ever need go beyond a quiet little town.. you make adjustments (as has already been said) to suit your situation.
I mean if we all knew when we would be attacked, we'd all opt to carry enough armament to scare the shorts off of Rambo that day or just not be there when it was supposed to happen. We don't, so we make judgement calls and assess things as we go along as to what to pack, what to keep near, what to leave in the safe that day.
The worst thing would be to get in a habit of just carrying one gun one way; with the awareness here, I'd say nobody on this board is likely to succumb to that, but people do get comfy with their habits. When in Rome, have something that should stop a Roman, and some basic plans.
While the .380 isn't what I'd prefer to have between me and eternity (I want a Seal Team between harm's way and my valuable hide!) it IS what I find I can carry everywhere in all weather, in the launching tube of a P3AT tucked into a pants or shorts pocket.
The .380 is my most basic loadout, but if I am likely to be in a bad area, or if my attire and the weather is friendly, you can bet there's a little something nice and comforting in a .40 S&W near abouts.