Prepare the cake: Preheat the oven to 350°F (175°C). Grease and flour two 8-inch round cake pans and set aside.
In a food processor or blender, crush the fresh mandarin oranges.
In a large bowl or stand mixer, cream together the softened butter and sugar until light and fluffy. Beat in the egg yolks, crushed mandarin oranges, and vanilla extract.
In a separate bowl, sift together the cake flour, baking powder, and salt.
Gradually add the dry ingredients to the wet mixture, alternating with the milk, until just combined. Do not overmix.
Pour the batter evenly into the prepared cake pans and bake for 25-28 minutes, or until the cakes spring back when touched lightly.
Let the cakes cool for 20 minutes before turning them out onto a wire rack or cake stand to cool completely.
For the Frosting + Filling:
In a food processor or blender, crush the fresh pineapple and set aside.
In a stand mixer, whip the heavy cream until stiff peaks form. Gradually add the powdered sugar, crushed pineapple, and the optional vanilla pudding mix. Continue beating until well combined.
Assemble the cake: Place one layer of the cooled cake onto a serving dish. Spread a layer of the pineapple cream frosting over the top. Place the second layer of cake on top, then ice the entire cake with the remaining frosting.
Garnish as desired and refrigerate the cake until ready to serve.