Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

  Touchstone Touchstone Functional Testing - Phase 1 - Test Plan

...

  • Stanford's WebAuth
  • Internet 2's Shibboleth
  • SAML (the Security Assertion Markup Language)
  • A new account management system for some users outside of the traditional MIT community
  • HTTP/S (extensive redirects)
  • SSL
  • MIT X.509 certificates
  • Kerberos (via the HTTP/SPNEGO protocol)
  • TLS
  • OpenID
  • Web Services
  • MySQL (including replication)
  • Apache
  • Tomcat
  • IDP High Availability Package
  • LDAP
  • KDC
  • DNS load balancing balancing

2.2 Scope

2.2.1 Items To Be Tested

...

The following modules and types of tests are considered to be outside the scope of this test effort and will not be tested by Questcon.   Testing may be performed by internal MIT personnel:

...

Risk

Contingency

QA environment not available

Utilize development or production environment

Insufficient time to fully test the Touchstone application with all major web browsers

Execute ~80% of application functionality with MIT's standard browser (Mozilla Firefox 2.0) and ~20% of the functionality with other browsers.

Test time increases due to changes in scope requiring additional test analysis and/or test case creation

If test time cannot be increased, reduce/cut overlap in multi-browser testing and execute highest priority test cases initially followed by lower priority tests until test time runs out

Excessive defect rate or defect correction time

Execute test cases in unrelated/unblocked functional areas of the application based on designated priority

Insufficient time to fully test theTouchstone the Touchstone application with all major web browsers

Prioritize the risk ofthe of the functionality to be tested.   Identify high risk functionality and ensurethat ensure that high risk items are tested thoroughly.   All high priority items willbe will be tested in both browsers.   As time permits additional functionality willbe will be tested based priority and risk with the most attention focused on theMozilla the Mozilla browser.

Test time increases due to changesin changes in scope requiring additional test analysis and/or test case creation

If test time cannot beincreasedbe increased, reevaluate priorities and risk and test according to new priorities.

Excessive defect rate or defectcorrection defect correction time

Extend testing windowif window if possible.   If not possible then execute test cases inunrelatedin unrelated/unblocked functional areas of the application based on designatedprioritydesignated priority.   Identify problem areas and ensure that additional attention isfocused is focused on these areas for regression testing.

...

The overall approach to this test effort will be to validate that the Touchstone UI meets the needs of the MIT to provide a mechanisim mechanism for single sign on using a federated id.   Validation will be performed based on test cases derived from the documented use cases, front-end functional designs, as well as exploratory testing heuristics.

MIT has indicated that the user community is largely standardized on Mozilla's Firefox v2.0 web browser in a PC or Mac environment.   Rather than re-executing all tests with each browser, Questcon will execute approximately 80% of the test cases using Firefox and approximately 20% of the tests using IE 6 and 7 (both in a PC/Windows architecture).   There will be some overlap in testing and touch points, but not enough to significantly impact the schedule.   Questcon will analyze the test cases to identify the best candidates for execution using IE 6 and 7.   IE 6 and 7 test cases will be chosen based on the amount of functionality traversed in the application.   In other words Questcon will attempt to "touch" as much of the application as possible using IE 6 and 7.  

Furthermore, a significant portion of the user community utilizes the Mac OS X operating system with the Safari browser.   Some duplicate testing will be performed by Questcon utilizing Safari and Firefox for the Mac (10% or less).   MIT should designate a group of users to execute more tests using the Safari/Mac and Firefox/Mac combination of browser and operating system.   Questcon will assist the users in identifying the best tests to execute.

...

Type of Testing

Included
Y/N

Comments

Team Responsible

Automation

N

MIT personnel will utilize jMeter to automate portions of the backend testing; no test automation tools will be used for the UI testing (use of Flash prevents this).

N/A

Conversion

N

There is no pre-existing system, ; therefore no data conversion is necessary.

N/A

Exploratory

Y

Some level of exploratory testing will be conducted based on heuristics related to typical rich-content internet applications.

Questcon

Functional

Y

Functional testing will be performed based on test cases derived on the documented use cases and front-end functional design.

Questcon

Installation / Upgrade

N

Because this is a web application no installation testing is necessary.

N/A

Integration

Y

Some integration testing will naturally occur as the front-end of the Touchstone application interfaces with and utilizes the back-end APIs.

Questcon

Parallel

N

There is no existing system that Touchstone is replacing.

N/A

Performance

N

Performance testing is coved by a separate Test Plan.

N/A

Regression

Y

Questcon expects to run at least a minimum regression test set prior to release to production.

Questcon

Security

Y

Backend security testing will be done by MIT.   Questcon will execute basic security/login testing on the front-end

MIT - Tester (backend)
Questcon (front-end)

UAT

Y

The user community will be tasked with performing ad-hoc user acceptance testing, domain specific metadata testing (metadata titles, tag lists, etc.),   as well as previously designated documented functional test cases for multiple browser/OS configurations (primarily Safari or Firefox/Mac configurations).

MIT - User Community

Unit

Y

Questcon expects the MIT developers to perform unit testing prior to releasing code to the test environment.

MIT - Developers

...

Hardware

O/S

Other

PC

Windows XP Professional SP 2

  • Mozilla Firefox v2+
  • Microsoft IE v6+
  • Adobe Flash v9+1

Macintosh PowerPC

Mac OS X

  • Firefox v2+
  • IE v6+
  • Flash v9+ 
  • Safari v2+

4.0 Schedule of Deliverables and Resources

...

This section identifies the deliverables, delivery date and resource responsible for each deliverable.

Key Deliverables

 Description Description

Expected Delivery Date

Resource

Test Plan

This document.

TBD

TBD

Test Case Designs

List of objectives and priorities of the tests.

TBD

TBD

Test Cases

Steps and expected results.

TBD

TBD

Status Reports

Accomplishments, issues and plans.

Weekly

TBD

Test Logs

Run logs.

Ongoing during test execution

TBD

Defect Reports

Entered in Jira as they are discovered.

Ongoing during test execution

TBD

Test Summary Report

Details the results of the testing effort.

TBD

TBD

...

The planned test schedule of the Touchstone project has an anticipated start date of //2008 and completion date of //2008.   The estimated completion date is based on several assumptions, some of which have been identified in 2.3 Risks & Contingencies. 

Milestone

Target Timeframe

Summation of Activities

Develop test strategy / plan

//2008 - //2008

  • Analyze existing design documents, notes, and other available materials
  • Develop test plan document

Review test plan

//2008 - //2008

  • Review, clarify, correct, and update the test plan
  • Client approval of test plan

Perform test analysis

//2008 - //2008

  • Develop test case design document

Review test case design

//2008 - //2008

  • Review, clarify, correct, and update the test case design

Build functional test cases / scenarios

//2008 - //2008

  • Combine test objectives into test cases
  • Document data, procedures, and results
  • Prioritize test cases
  • Determine which test cases will be executed in different browser/OS configurations

Setup test environment

//2008 - //2008

  • Setup web server and database server
  • Load application under test
  • Setup logins and authorizations

Setup test data

//2008 - //2008

  • Review & analyze test cases to target data to load in test environment
  • Load initial test data set

Execute functional & exploratory tests

//2008 - //2008

  • Execute documented test cases, as well as exploratory tests
  • Communicate with the development team when issues are found
  • Maintain a test run log
  • Track test metrics

Investigate / correct defects

//2008 - //2008

  • Investigate and validate that a defect has been found
  • Log defects in Jira
  • Work with the development team, as necessary, to identify the cause of the defect
  • Accept and retest defect corrections from the development team

Execute regression tests

//2008 - //2008

  • Execute a prioritized subset of test cases as regression of the system once all functional and exploratory testing is complete
  • Validate that no new errors have been introduced as a result of correcting known defects or configuration management / version control issues
  • Investigate and validate that a defect has been found
  • Log defects in Jira
  • Work with the development team, as necessary, to identify the cause of the defect
  • Accept and retest defect corrections from the development team

Execute UAT

//2008 - //2008

  • Work with the user community to identify and manage the execution of user acceptance tests
  • Communicate with the development team when issues are found
  • Investigate and validate that a defect has been found
  • Log defects in Jira
  • Work with the development team, as necessary, to identify the cause of the defect
  • Accept and retest defect corrections from the development team

Create test summary

//2008 - //2008

  • Create and deliver a test summary report to include:
    • Summation of planned/actual test activities
    • Deviation from planned activities
    • Summary of defects (open defects)
    • Summary of test metrics

...