Preheat the oven to 350 degrees. Spray a loaf pan with cooking spray.
In a large bowl cream together the butter and sugar.
Add in the eggs and mix well.
In a medium bowl combine the flour, baking powder, and salt. Mix well.
Add half of the flour mixture to the butter mixture and mix well.
Stir in the yogurt and vanilla extract. Add in the key lime juice (should be about 2 or 3 tablespoons) and key lime zest and mix well.
Add remaining flour and mix until just combined.
Pour into prepared loaf pan
Bake for 45-55 minutes or until a toothpick inserted into the center comes out clean.
Cool for 10 minutes then run a butter knife around the edges of the pan and turn onto a wire rack. Cool completely.
To make the glaze whisk together the powdered sugar and key llime juice until well combined. Drizzle over top of the cake. Top with sliced key limes if desired.