In a small saucepan, heat olive oil and garlic over medium heat. Once the cloves start to sizzle, turn the heat to medium-low and cook until the cloves are golden and tender (about 8 mins). Take the pan off the heat, transfer the cloves to a small bowl, then add a small sprig of rosemary to the hot oil. Let it sizzle for 30 seconds, then remove and discard the rosemary. Set aside 2 tablespoons of the infused oil to cool.
Add goat cheese and cream cheese to a food processor, and blitz to combine. Add chopped fried garlic, chopped rosemary, lemon juice, lemon zest, 1/2 teaspoon kosher salt and 1/2 teaspoon black pepper, pulsing to combine. With the motor running, stream in 2 tablespoons of the cooled infused oil. Season to taste with more salt and pepper if needed.
To serve, transfer dip to a bowl and top with a little more infused oil and a few rosemary needles. Enjoy with toasted bread and/or crackers.