Preheat oven to 350 degrees F. Line 2 muffin tins with about 16 paper liners. Set aside. Meanwhile, in a large bowl, prepare the cake batter according to package directions. Stir the box of chocolate pudding mix into the batter.
Portion the batter evenly among the cupcake tins, filling about 3/4 full. Bake for approx. 15-18 minutes or until a toothpick inserted near the center comes out clean. Cool the cupcakes completely.
In a large bowl, beat together the butter, marshmallow fluff and vanilla extract until creamy, about 2 minutes. Gradually add the powdered sugar, about one cup at a time, until light and fluffy. Add in the milk if frosting is too thick.
Pipe frosting onto cooled cupcakes and dredge the sides of the frosted cupcakes in sprinkles. Top with a Marshmallow Puff Cookie. Serve immediately, or store leftovers airtight at room temperature.