Combine the cream, milk, peanut butter and both sugars in a medium saucepan. Place the pan over medium heat and warm the mixture, whisking frequently, until the sugars dissolve and the peanut butter melts into the cream. It doesn't need to come to a boil—as soon as the mixture is smooth and homogenous, you can remove it from the heat. Whisk in the vanilla extract.
Pour it into a bowl or plastic container, and place it in the refrigerator until chilled. If you want to speed this process up, place the bowl over an ice bath and whisk it as it cools down.
Once the peanut butter mixture is chilled, pour it into your ice cream maker and churn according to manufacturer's directions, until it is the consistency of soft-serve ice cream. Stir in the miniature peanut butter cups by hand.
If you want to add a fudge and/or peanut butter swirl, spread about a third of the peanut butter ice cream in the bottom of a 9x5-inch loaf pan. Drizzle the peanut butter sauce over the ice cream, then top it with half the remaining ice cream. Drizzle the fudge sauce over this, and finish by covering it with the rest of the ice cream. (If you're omitting the swirls, simply spread all of the ice cream into the loaf pan.) Press a layer of cling wrap on top, and freeze the pan until the ice cream is firm, about 2 hours.