How do we assess if we are making a difference?

Evaluation of an initiative focused on change within an organization may include multiple desired outcomes. It is important at the beginning of the planning to identify methods for measuring successful achievement of identified goals. Options for format, methods, and frequency of measurements might include the following.

If an identified goal is to improve nurses knowledge of genomics, the use of a pre- and post- educational intervention survey may be appropriate.

•  Pre/Post Surveys
   

MINC sites used the Genetics and Genomics in Nursing Practice Survey (GGNPS) to:

1. Be able to assess the baseline knowledge, attitudes, practices, receptivity, confidence, and competency of nurses in their organization
    
2. Determine the impact of the use of their institution’s educational intervention on genomic knowledge, attitudes, practices, receptivity, confidence, and competency.
    

The GGNPS has been administered online or in paper format. The GGNPS is in the public domain and can be used as is or modified. Your institutional IRB may require approval before implementing the survey. For information about previous GGNPS study results, see the listing of journal articles.

Genetics and Genomics in Nursing Practice Survey Instrument Scoring Manual.

Genetics and Genomics in Nursing Practice Survey Instrument

•  Get Feedback Through Interviews or Forms
   

Feedback regarding satisfaction with the educational intervention can be solicited through a number of ways. Interviews with selected staff, collection of evaluation comments from all attendees, or conducting focus groups with staff to determine their perception of the offering could provide valuable comments for consideration. If the intervention is a web offering requesting feedback online can streamline the process of receiving feedback. Or from printed resources, a tear off card may help provide comments to guide suggested modifications.

If an identified goal is to have a specific practice change, there are several options to consider.

• Quality Improvement/Outcome Indicators
   

MINC sites developed quality improvement (QI) indicators based upon individual institution goals and objectives. Below is one example of a family history assessment quality improvement project with pre and post data associated with the goal of improving the collection of family health history information:

**(Genomics QI programs may help meet The American College of Surgeons (ACoS) Commission on Cancer (COC) accreditation standards for quality improvement processes.)

• Other Outcome Metrics that could be collected might include:

1. Pre- and post-educational intervention data on genetic referrals
    
2. Pre- and post-educational intervention data on community outreach programs
    

Process Evaluation is another method for assessing the initiative.

For example, addressing the “who, what, when, where” of the genomics program implementation.

• Did you implement the activities required for the genomics initiative?
   
• Who were the people involved?
   
• What problems were encountered?
   
• When was the initiative completed?
   
• What were the outcomes?
   
• How In the process, you may have established valuable genomic liaison groups --- an example of a serendipitous process evaluation outcome?
   

Reference: CDC: Types of Evaluation

Cost analysis projections by administrators for implementing a genomics competency initiative are recommended in the Administrative Section. Evaluation of the actual costs associated with a new competency initiative, and/or resource costs of integrating new genetic/genomic services, and/or costs of recommended infrastructure support for integrating genomics into practice (i.e., EHR and/or policy revisions) could help the administrator determine if their financial predictions were accurate for introducing genomics at their worksite. Plus, consideration of revenue resulting from changes associate with the new competency initiative may help support future decisions.

Administrator Proforma Example

Genetics and Genomics in Nursing Practice Survey (GGNPS)

Genetics and Genomics in Nursing Practice Survey Instrument Scoring Manual

NIH Clinical Center: Igniting Genetic and Genomic Education and Competency in a Research Facility: Successes and Challenges

Types of Evaluation