Preheat the oven to 350 degrees. Line a muffin tin with 12 cupcake liners and set aside.
In a medium bowl combine the flour, baking powder, and salt. Mix well and set aside.
In a large bowl combine the butter and sugar. Mix well until smooth.
Add in the eggs one at a time, beating after each addition until combined. Stir in the vanilla.
Place the milk in a microwave safe bowl and heat for 45 seconds. Remove from the microwave and add the Vanilla Chai tea bag. Steep for 5 minutes. Remove the tea bag.
Add the flour mixture to the butter mixture and mix well. Mixture will be thick.
Add in the milk and stir until a smooth batter forms.
Divide the batter between the cupcake liners filling each about 2/3 of the way full.
Bake for 20-25 minutes or until a toothpick inserted into the middle comes out clean. Cool completely on a wire rack.
Place the butter in a large bowl. Beat it with a mixer until smooth. Add in 2 cups of the powdered sugar and beat until smooth.
Place the milk in a microwave safe bowl and heat on high for 40 seconds. Add in the Constant Comment tea bag and steep for 5 minutes.
Remove the tea bag and add the milk and vanilla to the butter mixture and mix until smooth.
Add in the remaining powdered sugar a half cup at a time until desired consistency is reached.
Pipe the frosting on the cooled cupcakes and enjoy with a mug of your favorite Bigelow Tea.