In English learning, emotional factors are critical. In addition, emotional factors such as the students' learning potential, motivation, anxiety, self-esteem and others have important influence on English learning. For the English listening anxiety, academic world have studied on it in the last century eighty's. According to the study, it showed that the students' anxious psychological phenomenon performs significantly in the process of English listening and speaking and with the anxiety, students will produce barriers in English listening. In this paper, based on the clustering analysis, it takes further study on anxious self-schema of college students' English listening and type it. So as to study the differences of different anxious self-schema in school types and gender of college students and evaluate correctly the situation of English listening anxiety of college students.