Welcome!

The WAP evaluation team is wrapping up data collection from state weatherization offices and nonsampled agencies. To date:

• 97% of grantees have completed the S1 Program Information Survey
• 89% of local weatherization agencies have completed the S2 Program Information Survey
• 49% of sampled agencies have completed the S3 Detailed Program Information Survey
• 7% of sampled agencies have completed the DF4 Utility Information Survey

Data collection from sampled agencies will continue through early 2013. In November we began fielding the Utility Info Survey (DF4). This survey asks sampled agencies to provide client demographics and utility account information for a sample of homes weatherized in Program Year 2010 and 2011. This information will be used to characterize population of clients served by the weatherization program, and will also be used to request pre- and post-weatherization energy usage data from utilities. Agencies' work on this survey is a key step in the process of quantifying the energy impacts of weatherization in PY 2010.

The Performance Field Study is also underway, with field work continuing through late January. We are visiting 120 homes that were weatherized in Program Year 2008 in an effort to identify common factors that contribute to significantly higher or lower energy savings than the national average. Thirty six weatherization agencies are assisting us with this research.

After kicking off the ARRA phase of the national WAP evaluation in July, the weatherization community has made tremendous contributions to this data collection effort. To date:

• 78% of grantees have completed the S1 Program Information Survey
• 70% of local weatherization agencies have completed the S2 Program Information Survey
• 30% of sampled agencies have completed the S3 Detailed Program Information Survey

The evaluation team expects to wrap up survey data collection for grantees and nonsampled agencies by mid-November. We will be starting Utility Info Survey (DF4) requests for sampled agencies in the next few weeks.

We are also launching the Performance Field Study in October. This study will include site visits to 120 homes that were weatherized in Program Year 2008. The goal of this research is to identify common factors that contribute to significantly higher or lower energy savings than the national average.

Data collection has been underway since late July and the first wave of survey responses is rolling in. Thirteen grantees (22%) have completed the S1 survey, with many more working to meet late August deadlines. One hundred and eight (12%) local weatherization agencies have completed the S2 survey. Our team is impressed by the weatherization network's responsiveness to these data requests. We are very appreciative of the time and effort that everyone is putting forth to provide important information to the evaluation team. In September we'll be sending the DF4 Utility Info Survey to sampled agencies and continuing to support states and local agencies in their efforts to complete S1 and S2.

The webinars below let grantees and subgrantees know what to expect from the ARRA-period evaluation of the Weatherization Assistance Program. These presentations cover the scope and purpose of the evaluation, provide an overview of the data requests that will be made of grantees and subgrantees, answer FAQs, and provide an update on the expected timeframe for publication of retrospective WAP evaluation (PY2008) results. Click the links below to access these 20-minute presentations.

You will need to download Microsoft Silverlight to view.

• Webinar for grantees
• Webinar for subgrantees

Data collection for the ARRA-period evaluation of the Weatherization Assistance Program has begun. This research will quantify the energy savings, cost-effectiveness, and other benefits produced by the Weatherization Assistance Program during Program Year 2010. The scope includes weatherization activities funded by regular WAP allocations from DOE as well as ARRA, SERC and WIPP grants. Results will be influential in decisions about future program design and funding. Full participation by all grantees and subgrantees will ensure robust results.

Grantees will be asked to complete one data request for the evaluation, the S1 Program Information Survey. This survey requests fiscal information on DOE and leveraged funds administered during Program Year 2010. The survey also addresses state-level training and monitoring activities, as well as aspects of program implementation during the ARRA period.

All local weatherization agencies that received DOE funding in PY 2010 will be asked to complete the S2 Program Information Survey. This survey is similar to S1, requesting information on funds expended and units produced, as well as details about ARRA-period weatherization activities.

Four hundred and fifty local agencies were randomly sampled to provide additional data to the evaluation team. The sample is based on PY 2010 production, meaning that sampling likelihood increases with higher production. In addition, all agencies that received SERC grants were sampled.

The evaluation team will be working closely with sampled agencies over the next nine months. Sampled agencies will receive at least four survey requests and may be subject to additional requests for field studies. The data provided by sampled agencies is of critical importance to measuring program impacts such as energy savings and cost-effectiveness. We recognize that these requests are coming during a time of funding cutbacks and staffing reductions. Our team of case managers will do as much as they can to provide flexibility and support to make this process as easy as possible.

The WAP evaluation team is gearing up to begin the ARRA-period evaluation of the Weatherization Assistance Program. This study will measure the impact of weatherization during the ARRA period (PY 2009, 2010 and 2011), with a focus on Program Year 2010. The scope includes weatherization funded by regular WAP allocations from DOE as well as ARRA, SERC and WIPP grants. The ARRA-period data collection effort will begin in early 2012 once authorization from the Office of Management and Budget is received.

For state and local agency staff who participated in the retrospective WAP evaluation (PY 2008), the data requests for the ARRA-period evaluation will look quite familiar. By fielding a similar set of surveys, we will be able to compare the results achieved under the ARRA period with the pre-ARRA baseline measured during the retrospective evaluation.

If you work for a state weatherization office or local agency, the evaluation team will send you detailed information about what you'll be asked to do once OMB approval is received. In the meantime, you can view the draft survey instruments and presentations about the ARRA-period evaluation.

And if you're interested in results from the retrospective WAP evaluation, ORNL will begin publishing reports and key findings in 2012. We will link to these reports as they are published.

Nearly 30 percent of sampled agencies have completed the Housing Information Survey (DF2/3), the final survey for the pre-ARRA evaluation. We greatly appreciate agencies' prompt response to this time-intensive data request. With data collection complete for the two program information surveys (S2 and S3), we are pleased to report the following response rates: 92% of all agencies completed S2 and 89% of sampled agencies completed S3.

In April the evaluation team will begin contacting sampled agencies to request assistance with recruiting clients for the Occupant Survey (S4). This survey asks WAP clients about their experiences with the program and collects data that will be used in the assessment of non-energy benefits of weatherization. We will ask agencies to identify 5-10 clients that are eligible for weatherization services, but no audit or weatherization work has been done yet. We will ask for the client's name and contact information, and request that the agency hold off on conducting the audit for approximately two weeks. We will administer three surveys to participating clients: (1) prior to audit & weatherization; (2) shortly after weatherization; and (3) one year after weatherization. The client will receive an incentive upon completion of each survey.

The evaluation team is rolling out the Housing Information Survey (DF2/DF3) to sampled agencies. The survey collects detailed information about a sample of homes and buildings weatherized in Program Year 2008. For most agencies, this will be the final request we will make as part of the pre-ARRA evaluation. It is also an essential piece of the evaluation that will allow us to document energy savings from weatherization. This information will be key for DOE and policy-makers making future decisions about program funding and scope.

As the survey portion of the evaluation nears its conclusion we are pleased to report the following participation results: 97% of all agencies have completed the DF10 survey, 86% of all agencies have completed the S2 survey, 83% of sampled agencies have completed the DF4 survey, and 75% of sampled agencies have completed the S3 survey. We thank all agencies for their responses to evaluation data requests. Please note that February 4th is the final date for submitting S2 and S3.

APPRISE is kicking off the process evaluation field study in January. Twenty agencies have been randomly selected to participate. The study will focus on the weatherization process and client interaction, and will involve 4-5 weeks of detailed observation of agency practices by a weatherization expert and a social scientist. These researchers will observe service delivery and document how services are delivered to clients. Findings across all participating agencies will be aggregated and summarized for the evaluation. Specific information about participating agencies will not be shared with any other agency or office.

The indoor air quality study kicked off in early November, with additional installation rounds scheduled for December and January. The study is measuring the effect of weatherization on radon, carbon monoxide, formaldehyde and humidity in single family and mobile homes. It will involve a total of 640 homes in 33 states. We are also wrapping up recruitment for a fuel oil metering study that is measuring pre-and post-weatherization heating energy usage in 128 single family homes in the Northeast. A similar propane study is metering 32 homes in the Midwest and South this fall with an additional 32 homes slated for fall of 2011. The evaluation team is grateful to participating agencies for their assistance in identifying candidate homes for these important field studies.

Agencies continue to make good progress in completing evaluation surveys. Eighty four percent of nonsampled agencies have completed the survey portion of the evaluation. DOE has set a final deadline of November 30th for nonsampled agencies to complete the S2 Program Information Survey. November 30th is also the DOE deadline for sampled agencies to complete the DF4 Utility Info Survey. Sixty percent of sampled agencies have finished DF4.

Fuel metering studies for homes heated with propane or fuel oil are ramping up, with the indoor air quality study soon to follow. The evaluation team will be contacting agencies that have been sampled for these studies over the next few weeks. We hope agencies will support this important research by helping us identify candidate homes. On the survey front, over 60 percent of nonsampled agencies have completed the survey portion of the evaluation. The short-term priority for sampled agencies is to obtain the data needed to request billing histories for weatherized homes (DF4 survey). The evaluation team continues to be impressed with agency responsiveness to our data requests!

State data collection is mostly wrapped up aside from a few stragglers still working on their last survey. The evaluation team would like to thank all the states that have fulfilled their component of the evaluation. Eighty percent of all agencies are done with their first survey and about a third of nonsampled agencies are finished with the evaluation. In September the focus will be on data collection from sampled agencies. The evaluation team thanks sampled agencies for their ongoing hard work.

More than 90 percent of states have completed the first of two state surveys, and the evaluation team is looking to wrap up the state data collection process by early August. Over 200 agencies have completed the first agency survey and 23 agencies have finished all required surveys. The evaluation team appreciates the responsiveness that participants have shown to date.

The WAP evaluation's state and agency data collection effort has gotten off to a great start. A little more than a month after the evaluation team's initial outreach to state weatherization offices, 25 states have completed the first of two data requests and two states have finished both surveys. Twenty two weatherization agencies have completed the first agency survey. The evaluation team would like to thank all participants for their efforts.