This paper discusses the finite groups with the number of conjugate classes of acyclic subgoups less than or equal to 2, and gives the complete classification of the groups.