In a medium bowl, beat together the softened cream cheese, powdered sugar, 2 tablespoons milk, and half the vanilla extract until smooth. Gently fold in 1 cup of blueberries, 1 tablespoon lemon juice, and a pinch of lemon zest. Set aside.
Grease a 9x13-inch baking dish generously. Spread half of the bread cubes evenly across the bottom. Dollop the cream cheese mixture over the bread and spread gently. Top with the remaining bread cubes, pressing down lightly.
In a large bowl, whisk together the eggs, 2 cups milk, remaining vanilla, cinnamon, nutmeg, remaining lemon zest, and 1 tablespoon lemon juice until well combined. Pour slowly over the bread layers, pressing down gently to help absorption. Scatter the remaining cup of blueberries on top.
Cover tightly with aluminum foil and refrigerate for at least 8 hours or overnight.
When ready to bake, remove from fridge and let sit at room temperature while preheating oven to 375°F. Bake covered for 45 minutes, then uncover and bake an additional 30 minutes until golden brown and set in the center. Let cool 10 minutes before serving.