The performance evaluation of network security system generally requires combining vulnerability scanning technology and expert system in order to better discover security vulnerabilities existing in the system, also conducive to a comprehensive assessment of the overall network security, and be able to offer a complete and improved security reports. With the rapid development of network technology, requirements on network security have become more and more sophisticated, but related mature testing methods have been behind the overall growth rate, therefore, there is a need to establish a new set of performance evaluation for network information security. This paper proposes an evaluation of separating the control and experiment, establishes a new two-tier testing system of network information security, whose test results can provide a better theoretical basis and reference value for actual design and evaluation. First, the study reviews the current testing methods at home and abroad. Then it explains the working principle, architecture platform and application of this system, and studies on the safety requirements of the components and application of environmental analysis. At last, overall performance evaluation is made according to the actual test results, testing methods are verified to meet the requirements, and the entire test methods are summarized. From the verification, it can be seen that the testing program has a repeatable, fast and automotive features, and can help testers design more rational and effective test environment, besides, the entire testing process is very simple and test efficiency is very high.