Mash the banana: Peel a ripe, spotty banana and mash it well with a fork in a small saucepan until mostly smooth with a few small lumps.
Cook the banana syrup: Add the brown sugar, cinnamon and nutmeg to the mashed banana and cook over medium heat, stirring often, until the mixture thickens and turns glossy, about 3 to 4 minutes.
Cool the syrup: Remove the pan from the heat and let the syrup sit for about 5 minutes so it isn't hot enough to melt your ice right away.
Fill the glasses with ice: Divide the ice cubes between two tall glasses, filling them about three-quarters full.
Spoon in the syrup: Spoon the cooled banana syrup over the ice in each glass, dividing it evenly between the two.
Add the chai concentrate: Pour the chai tea concentrate over the ice and syrup, splitting it evenly between the two glasses.
Pour in the milk: Top each glass with whole milk, leaving a little room at the top for whipped cream if using.
Stir to combine: Stir each drink with a long spoon until the banana syrup swirls through and the color turns a light caramel brown.
Add whipped cream: Top each glass with a layer of whipped cream if you want the coffee shop finish.
Garnish and serve: Sprinkle crumbled banana bread and a dusting of ground cinnamon over the whipped cream, then serve with a straw.