Preheat your oven to 375°F (190°C). Grease a 12-cup muffin tin or line it with paper liners to prevent sticking.
In a large mixing bowl, whisk together the all-purpose flour, baking powder, baking soda, ground cinnamon, ground nutmeg, and salt. Set aside.
In another bowl, whisk the eggs, granulated sugar, brown sugar, vegetable oil, pumpkin puree, milk, and vanilla extract until smooth and well combined.
In a small bowl, mix 1/4 cup brown sugar with 1 tablespoon ground cinnamon. Set aside.
Pour the wet ingredients into the dry ingredients and gently fold with a spatula until just combined. Avoid overmixing to keep the muffins tender. If using nuts, fold them in now.
Spoon half of the muffin batter into the prepared muffin cups, filling each about halfway. Sprinkle a teaspoon of the cinnamon sugar mixture over each muffin. Top with the remaining batter, dividing it evenly. Use a toothpick or skewer to gently swirl the cinnamon sugar through the batter, creating that signature cinnamon roll effect.
Bake for 18-22 minutes or until a toothpick inserted in the center comes out clean. The muffins should be golden on top and spring back lightly when touched.
While muffins cool slightly, beat the softened cream cheese with powdered sugar until smooth and creamy. If the glaze is too thick, add a teaspoon of milk to thin it out.
Once the muffins are cool enough to handle, spread or drizzle the cream cheese glaze over the tops. Enjoy warm or at room temperature.