In this paper we measure the accuracy of password strength meters (PSMs) using password guessing resistance against off-the-shelf guessing attacks. We consider 13 PSMs, 5 different attack tools, and a random selection of 60,000 passwords extracted from three different datasets of real-world password leaks. Our results show that a significant percentage of passwords classified as strong were cracked, thus suggesting that current password strength estimation methods can be improved.