The process of musical emotion analysis is a rather complex analysis activity. It needs effective understanding of its musical emotion characteristics thus to make analysis thoughts maintain accurate accordingly which has significant impact on this study. Besides, the other key part is to conduct effective analysis and processing by model construction, and then the study process would be more scientific and reasonable. From the point of view of appreciators, factors of the formation of musical emotion is rather complex which needs constant improvement of the model. In this way, the analysis process of musical emotion can be strengthened real-timely. Combined with this principal line, this study conducts research and discussion. Among it, specific arrangements would be made through corresponding mathematical matrix model, which can provide solid theoretical and data basis. Meanwhile, this study will do effective dissertation of specific procedures of model construction to bring solid theoretical basis for the constant improvement of the practical value of the model.