Preheat the oven to 400°F. Gather a large oven-safe skillet and the listed ingredients so quantities are easy to follow.
Make the mustard marinade: In a bowl combine 2 tablespoons extra virgin olive oil, 2 tablespoons Dijon mustard, 3 of the chopped garlic cloves (reserve the remaining chopped clove for the orzo), 1 of the chopped shallots (reserve the other shallot for the orzo), 2 tablespoons chopped fresh rosemary, 1 teaspoon smoked paprika, and a pinch of kosher salt and black pepper. Add the 2 pounds boneless chicken breasts or thighs and toss to coat. Set aside to rest while you heat the pan (5–10 minutes).
Sear the chicken: Heat 1 tablespoon extra virgin olive oil in the large oven-safe skillet over medium-high heat. Add the chicken (work in batches if needed so pieces don’t crowd the pan). Sear until golden, about 3–5 minutes per side. During the last 2 minutes of searing, add 3 tablespoons salted butter and the 1 lemon, sliced, to the skillet so the slices brown lightly. Transfer the seared chicken and lemon slices to a plate, reserving any pan juices in the skillet.
Toast the orzo and aromatics: Reduce the heat to medium. Add the 1 1/2 cups orzo, the reserved chopped shallot, and the remaining 1 chopped garlic clove to the skillet with the browned bits. Cook, stirring, until the orzo is fragrant and lightly toasted, about 2 minutes.
Add liquid and kale: Pour in 3 cups low-sodium chicken broth and add 2 cups chopped kale to the skillet. Season with kosher salt and black pepper to taste. Increase the heat to high and bring the mixture to a boil.
Combine and bake: Slide the seared chicken and lemon slices (and any juices from the plate) back into the skillet, nestling the chicken into the orzo. Transfer the skillet to the preheated oven and bake uncovered for about 15 minutes, or until the chicken is cooked through (internal temperature 165°F) and the orzo is tender. Remove from the oven and let rest 3–5 minutes.
Make the feta sauce: While the skillet is in the oven, combine 4–6 ounces feta cheese, 1/4 cup plain Greek yogurt, the separate 1 clove garlic (grated), and the juice from 1 lemon in a blender or food processor. Blend until smooth and creamy. Stir in 1/4 teaspoon smoked paprika and a pinch of crushed red pepper flakes; adjust the red pepper flakes to taste.
Serve: Spoon the feta sauce over the chicken and orzo. Sprinkle with chopped fresh dill and additional cracked black pepper or red pepper flakes if desired.