Justice was originally a spirit in the Fade that embodies - you guessed it - justice. Because of some magic gone awry, he was taken out of the Fade and thrown into the human world, stuck in the body of a dead Grey Warden. In Dragon Age: Origins - Awakening, he met Anders and they became friends, and Justice urged Anders to take action against the Circle of Magi and their unfair and cruel actions. Anders did not seem too enthusiastic at first, but eventually he agreed to help Justice and become a host for him.
Unfortunately, due to the amount of anger that Anders had inside him for the Circle, it warped Justice into a spirit of Vengeance, and he is something of a second personality that Anders has (though Anders insists that they are "one", the dominant personality can change depending on the circumstances). When he witnesses injustices done to mages, or gets extremely angry, Justice will take over. He is difficult for Anders to control, and he is extremely dangerous and powerful. However, there is no known way for him to be separated from Anders.
Anders mentions that he hears Justice's thoughts as his own, implying that his head is probably a place you never want to go, because it's very confusing and full of crazy.