Heat the oven to 350°F (175°C) and line a 12-cup muffin pan with paper liners. Add the cake mix, eggs, vegetable oil, water, and peanut butter to a large bowl, then beat until smooth according to the cake-mix package directions for mixing time.

Fill each paper liner about halfway with batter. Place approximately 1 teaspoon of strawberry or grape jelly in the center of each cupcake, keeping the filling away from the sides of the liner.

Spoon the remaining batter over the jelly until each liner is about two-thirds full. Bake for 18–22 minutes, until the tops spring back lightly and a toothpick inserted into the cake portion comes out clean.

Cool the cupcakes in the pan for 5 minutes, then transfer them to a wire rack. Let them cool completely before frosting so the buttercream stays thick and does not slide.

Beat the softened butter and peanut butter until creamy. Gradually add the confectioners' sugar, then beat in the vanilla, optional salt, and enough milk or heavy cream to create a smooth, spreadable frosting.

Frost the completely cooled cupcakes with a knife or piping bag. Finish with a small spoonful or drizzle of additional jelly for a colorful decoration, then serve.