Section 455(b) (1) provides that a judge should disqualify himself in any proceeding in which he has “a personal bias or prejudice concerning a party, or personal knowledge of disputed evidentiary facts concerning the proceeding.” 
Section 455(b)(2) mandates that a judge must disqualify himself when “in private practice he served as lawyer in the matter in controversy, or a lawyer with whom he previously practiced law served during such association as a lawyer concerning the matter, or the judge or such lawyer has been a material witness concerning it.”
As for §455(b)(3), disqualification is currently required whenever the judge “has served in governmental employment and in such capacity participated as counsel, adviser or material witness concerning the proceeding or expressed an opinion concerning the merits of the particular case in controversy.”
Pursuant to 28 U.S.C. §455(b) (4), a judge must disqualify himself if he knows that he, individually or as a fiduciary, or his spouse or minor child residing in his household, has a financial interest in the subject matter in controversy, or in a party to the proceeding, or any other interest that could be substantially affected by its outcome.
Finally, § 455 (b)(5)(i) prescribes judicial disqualification whenever the judge, his spouse, a person within the third degree of relationship to either of them, or a spouse of such a person is a party to the proceeding or an officer, director, or trustee of one. see here.



These are the reasons given for recusal.  If an party in a case believes one of these 5 reason apply to a judge, they can make a motion to have the judge/justice recused/dismissed. The DOJ has a Dept of Ethics which evaluates the motion and gives written permission to make the motion if by a government attorney.