Welcome to NYCU CSIT Mirror site
10. A GUIDELINE ON SECURITY TESTING
These guidelines are provided to give an indication of the extent and sophistication of testing undertaken by the DoD Computer Security Center during the Formal Product Evaluation process. Organizations wishing to use "Department of Defense Trusted Computer System Evaluation Criteria" for performing their own evaluations may find this section useful for planning purposes.
As in Part I, highlighting is used to indicate changes in the guidelines from the next lower division.
10.1 TESTING FOR DIVISION C
Personnel
The security testing team shall consist of at least two individuals with bachelor degrees in Computer Science or the equivalent. Team members shall be able to follow test plans prepared by the system developer and suggest additions, shall be familiar with the "flaw hypothesis" or equivalent security testing methodology, and shall have assembly level programming experience. Before testing begins, the team members shall have functional knowledge of, and shall have completed the system developer's internals course for, the system being evaluated.
Testing
The team shall have "hands-on" involvement in an independent run of the tests used by the system developer. The team shall independently design and implement at least five system-specific tests in an attempt to circumvent the security mechanisms of the system. The elapsed time devoted to testing shall be at least one month and need not exceed three months. There shall be no fewer than twenty hands-on hours spent carrying out system developer-defined tests and test team-defined tests.
10.2 TESTING FOR DIVISION B
Personnel
The security testing team shall consist of at least two individuals with bachelor degrees in Computer Science or the equivalent and at least one individual with a master's degree in Computer Science or equivalent. Team members shall be able to follow test plans prepared by the system developer and suggest additions, shall be conversant with the "flaw hypothesis" or equivalent security testing methodology, shall be fluent in the TCB implementation language(s), and shall have assembly level programming experience. Before testing begins, the team members shall have functional knowledge of, and shall have completed the system developer's internals course for, the system being evaluated. At least one team member shall have previously completed a security test on another system.
Testing
The team shall have "hands-on" involvement in an independent run of the test package used by the system developer to test security-relevant hardware and software. The team shall independently design and implement at least fifteen system-specific tests in an attempt to circumvent the security mechanisms of the system. The elapsed time devoted to testing shall be at least two months and need not exceed four months. There shall be no fewer than thirty hands-on hours per team member spent carrying out system developer-defined tests and test team-defined tests.
10.3 TESTING FOR DIVISION A
Personnel
The security testing team shall consist of at least one individual with a bachelor's degree in Computer Science or the equivalent and at least two individuals with masters' degrees in Computer Science or equivalent. Team members shall be able to follow test plans prepared by the system developer and suggest additions, shall be conversant with the "flaw hypothesis" or equivalent security testing methodology, shall be fluent in the TCB implementation language(s), and shall have assembly level programming experience. Before testing begins, the team members shall have functional knowledge of, and shall have completed the system developer's internals course for, the system being evaluated. At least one team member shall be familiar enough with the system hardware to understand the maintenance diagnostic programs and supporting hardware documentation. At least two team members shall have previously completed a security test on another system. At least one team member shall have demonstrated system level programming competence on the system under test to a level of complexity equivalent to adding a device driver to the system.
Testing
The team shall have "hands-on" involvement in an independent run of the test package used by the system developer to test security-relevant hardware and software. The team shall independently design and implement at least twenty-five system-specific tests in an attempt to circumvent the security mechanisms of the system. The elapsed time devoted to testing shall be at least three months and need not exceed six months. There shall be no fewer than fifty hands-on hours per team member spent carrying out system developer-defined tests and test team-defined tests.
Previous Next Table of Contents