Pour milk into a saucepan over low heat. Once milk is warm but not boiling (4-5min) add the chai tea bag and let simmer 5-7 minutes then remove from heat. Remove tea bag.
Preheat oven to 375F.
Butter a 9 x 13 baking dish and layer in torn bread pieces. Sprinkle half of the shredded coconut and half of the slivered almonds over the bread.
In a medium bowl, combine 1 ¼ cups heavy cream, eggs, white granulated sugar, and ¼ cup brown sugar. Whisk together until mixture thickens. Add chai mixture and mix until combined.
Pour chai cream mixture over bread slices, making sure bread is fully coated and soaking up the mixture. Cut 2 tbsp butter into small cubes and scatter around bread pieces. Top with the remaining coconut and almond.
Bake for 35-40 minutes, turning dish half way through for even browning. If bread pudding is browning too soon, tent with foil then continue cooking.
Prepare Caramel Sauce:
In a small bowl combine ¼ cup heavy cream, ginger and vanilla. Let sit for 10 minutes. In a sauce pan melt 2 tbsp butter with 1/4 cup brown sugar over low heat. When the sugar mixture has liquefied, add the cream mixture. Stir occasionally for 5 to 7 minutes until mixture thickens. Remove from heat; allow sauce to thicken.
Remove bread pudding from oven and serve immediately topped with caramel sauce.
Recipe adapted from: http://nygirlcaneat.com