Arrange a rack in the middle of the oven and heat to 350°F. Line a 9×13-inch baking pan with a parchment paper sling that hangs over the 2 long sides. Coat with cooking spray; set aside.
Make the Streusel:
Stir the flour, walnuts, brown sugar, cinnamon, and salt together in a medium bowl.
Incorporate the chilled, cubed butter with your fingertips until you have small- to medium-sized lumps and none of the flour-sugar mixture remains dry; set aside.
Make the Cake:
Whisk the flour, sugar, baking powder, and salt together in a large bowl.
Add the eggs, oil or butter, milk, and vanilla and mix with an electric hand mixer on medium speed until smooth, about 3 minutes.
Pour half of the batter into the pan and spread into an even layer. Sprinkle half of the streusel over the batter.
Pour in the rest of the batter evenly over top, then distribute the remaining streusel over the top.
Bake until the cake is lightly browned and springs back when gingerly pressed with a finger, 30 to 40 minutes.
Cool on a wire rack for 15 minutes. Grasping the parchment paper sling, remove the cake from the pan and cool completely before glazing.
Stir the powdered sugar and milk together in a small bowl until smooth. Drizzle the glaze over the cooled cake. Wait 10 minutes for the glaze to harden before cutting and serving.