User blog comment:LokiIsAwesome/Agravaine and a few other Series 4 characters.../@comment-5350801-20120815231736/@comment-4650675-20120824152951

I have two theories for your question, Fimber. The first is the obvious one: Uther's emotional state. After the events of series three, he became a mentally ill person, who cared for nothing happening around him. He did not communicate at all with Guinevere(at least it was not seen) who was taking care of him, despite their bad relationship in the past, and not even with his old friend, Gaius. I believe that his only worry, after Morgana's betrayal, was Arthur's well-being and he did not care for himself at all anymore. Given that Arthur is Agravaine's sister, everyone would assume that Agravaine would love Arthur, especially since he even made a promise with her.

The second theory is that Agravaine may have told Uther that he has forgiven him, and Uther, thinking that this was Ygraine forgiving him herself, felt comfortable having Agravaine around the court and may even have thought of him as a friend and an ally to himself and Camelot. However the first is more logical in my opinion :)