Heat a few inches of oil in a large saucepan over medium-low heat until it reaches 350°F (175°C). Line a plate with paper towels to drain the fritters after frying.
In a large bowl, whisk together the flour, brown sugar, baking powder, salt, cinnamon, and nutmeg.
In a separate small bowl, whisk together the milk, eggs, melted butter, and vanilla extract. Stir the wet ingredients into the dry mixture until just combined. Fold in the chopped apples.
Once the oil is hot, drop 1/4 cup of batter into the oil, frying only 1-2 fritters at a time. Fry for 3-4 minutes, flipping once or twice, until golden brown on both sides. Use a slotted spoon to remove the fritters and drain on the prepared plate with paper towels. Repeat with the remaining batter.
For the glaze, whisk together the powdered sugar, milk, and vanilla extract until smooth. If too thick, add more milk 1 teaspoon at a time until the consistency is perfect for dipping.
Once fritters have cooled slightly, dip each one in the glaze and place them on parchment paper. Immediately sprinkle with flaked sea salt.
Serve warm and enjoy!