Preheat the oven to 350 F. Generously grease a bundt pan and dust it with flour, then tap out the excess flour.
Combine the ingredients for the cinnamon swirl in a food processor and process them until they're in fine crumbs. Set aside for now.
Whisk together the pumpkin, buttermilk, and vanilla in a small bowl, and in a separate bowl, sift together the flour, baking powder, baking soda, salt, and spices.
Combine the butter and both sugars in the bowl of a large mixer fitted with a paddle attachment. Beat on medium-medium high speed until light and fluffy, for about 3-5 minutes. Add the eggs one at a time, beating well after each addition. Turn the mixer speed to low and add a third of the flour, then when that's almost mixed in, add half of the pumpkin mixture. Continue to alternate adding wets and drys, ending with the dry ingredients. When just a few streaks of flour remain, turn the mixer off and finish mixing by hand, scraping down the bottom and sides with a rubber spatula.
Spoon about a third of the batter into the bundt pan and smooth it out. Add half of the cinnamon swirl mix, then top with more batter. Add the rest of the cinnamon mix, and finish by spreading the rest of the batter on top. Bake the cake for about 45 minutes, until a toothpick inserted into the center comes out clean.
Cool the cake on a rack for 15 minutes, then invert it out of the pan and cool completely before frosting.
To Make the Brown Sugar Frosting:
Melt the butter in a small saucepan over medium heat. Stir in the brown sugar and cook, stirring, until the sugar melts. It might look separated, but that's okay. Stir in the cream, and it should all come together. Bring the mixture to a boil, then pour it into a mixing bowl fitted with a paddle attachment to cool for 10 minutes.
After 10 minutes, add the powdered sugar, vanilla, and salt. Beat well on medium speed for 30-45 seconds, until there are no lumps of sugar and the frosting is satiny and smooth. If the mixture appears too thin, add more confectioners' sugar, if it's too thick, add a spoonful of hot water. Be aware that it sets up QUICKLY, so as soon as you're finished mixing it, pour it over the cake while it's still warm. Know that it's hard to go back and correct mistakes because it starts setting and crusting soon after being poured, so one option is to put it in a large Ziploc bag and cut off a corner. Pipe the frosting over the top so it drips down where you want it to. Add any toppings or decorations before the frosting crusts over.