In a bowl, whisk together the olive oil, Dijon mustard, garlic, shallots, rosemary, smoked paprika, salt, and pepper. Coat the chicken in this mixture and marinate for at least 15 minutes.
In a large skillet, melt the butter over medium-high heat. Add the marinated chicken and sear for about 5-7 minutes on each side until cooked through.
In the same skillet, add the orzo and toast for 1 minute. Pour in the chicken broth, bring to a simmer, cover, and cook for about 10 minutes until the orzo is al dente.
Stir in the kale, return the chicken to the skillet, place lemon slices on top, cover, and let simmer for another 5 minutes.
For the feta sauce, combine feta, Greek yogurt, grated garlic, lemon juice, and smoked paprika in a bowl. Mix until smooth, adding water if needed.
Remove the skillet from heat, drizzle feta sauce over the chicken and orzo, and sprinkle with dill and red pepper flakes before serving.