What HOPES taught us: A summary of our four year-long evaluation of community-based child maltreatment prevention programs
By Swetha Nulu, MPH, Assistant Director of Research
Major Take-Aways
|
In 2015, the Texas Institute for Child & Family Wellbeing began a program evaluation for Project HOPES (Healthy Outcomes through Prevention and Early Support), a community-based child maltreatment prevention program in Texas implemented by the Prevention and Early Intervention division of Texas Department of Family and Protective Services. Over the past 4 years, our TXICFW team traveled to over 22 different counties in Texas to learn from program participants, providers, and stakeholders about Project HOPES, in what we liked to call the “Tour of Texas.”
Throughout this evaluation, we learned a lot about child maltreatment prevention in Texas, the countless ways staff support families and their communities, and a thing or two about multi-site, multi-intervention program evaluation.
In our Final HOPES Evaluation Report, you’ll find all the details of the program evaluation of Project HOPES, the evidence-based programs implemented, methodology, and the short-term, intermediate, and long-term outcomes evaluated. In this blog, you’ll find some highlights from our evaluation — for those looking for a deeper dive, check out the full report!
Creating our Evaluation Design
Fig. 1: Here are all the different Evidence-Based Programs that were implemented at HOPES sites from 2015-2019.
A challenging part of conducting the evaluation of Project HOPES was figuring out a way to evaluate 21 different evidence-based programs that had different curriculums, program lengths, and intended goals. Typically, program evaluations are focused on only one program, so it is more clear on how to evaluate the impacts of an intervention when it is implemented consistently.We decided to adapt the Centers for Disease Control and Prevention’s framework of community-based child maltreatment prevention that focuses on improving safety, stability, and nurturing across the child’s environment. Our research team used this framework as foundation for a conceptual framework tailored to HOPES programming that includes the evidence-based programming and the work done by the community coalitions.
Fig. 2: A visual representation of the different building blocks for safe, stable & nurturing relationships.
For this evaluation of Project HOPES, we aimed to assess HOPES programming related to safety, stability, and nurturing across all levels (child, caregiver, community, and policy) presented in the conceptual framework.
Understanding Child Maltreatment Rates in Texas
The main goal of the HOPES program was to prevent child maltreatment in Texas. County-level changes in child maltreatment rates take time and any changes in child maltreatment rates observed cannot be solely linked to HOPES programming, since the number of families served is often much smaller than the county’s population of families with a child. Additionally, greater community awareness of child maltreatment may also result in increased reporting and show an increase of child maltreatment rates. The map below shows the change of child maltreatment rates from 2010 to 2018 using Texas county-level CPS data of confirmed maltreatment cases of children ages five and younger.
Fig. 3: Total number of families reached across HOPES sites from 2015-2018.
In order to better understand the impact of HOPES programming on preventing child maltreatment, we compared families who participated in long-term evidence-based programs and found…
|
Again, it is important to emphasize that there are many different factors that we were unable to account for in the linked data of HOPES participants and CPS case after program participation. For example, families who are able to participate in long-term programming may not be at a high risk of child maltreatment compared to a family whose primary caregiver has a substance abuse problem and is unable to participate in HOPES.
The “Big 4” Factors of Child Maltreatment: Substance Use, Domestic Violence, Mental Health, and History of Trauma
As researchers and social workers in the field of child welfare, identifying families who are at the greatest risk for child maltreatment often focuses on four main risk factors, which we have coined as the “Big[MK15] [NS16] 4.” These risk factors are: caregivers’ substance use, history of domestic violence/abuse, mental health concerns, and history of trauma. Countless studies have shown that families who are experiencing one or more of these issues are at a greater risk of child maltreatment compared to families who do not have these issues.
If you’re interested in reading more about the “Big 4,” click here to read another blog we wrote about it.
Relationships are key to program involvement & time spent on case management services help build relationships
During our interviews with families receiving programming, a theme that consistently came up was how important their relationship was with their parent educator. While nearly all program participants valued the curriculum, many articulated that they were motivated to stay in the program because of the relationship and bond they had with their parent educator. Regardless of the curriculum, the relationship that families had with their parent educator was attributed to them participating in the program and applying the information in the curriculum to their daily lives.
Another theme we heard from families was how much they valued case management services, which ranged from being able to provide transportation to walking alongside a healthcare appointment to helping them apply to school or a job. Many caregivers discussed how the time HOPES staff spent on finding resources with them showed that their parent educator/case managers really cared for their family. This in turn helped families build trust with the staff, and the program as a whole.
While staff struggled balancing the curriculum and case management, our evaluation suggests that time spent on case management is necessary to help build trust and solidify bonds between families and the program—which ultimately results in lasting changes for families.
| [P]robably the most useful thing was helping us fill the gap in the formula when I was unemployed–and even though it didn’t quite pan-out, [my parent educator] trying to help me get back employed, that helped a lot…having that little bit of help from someone else going, “Hey, you know, we got this over here; you can try this,” that really helped a lot.”
– Parent, HOPES Participant |
Community collaborations are successful but require consistent communication and strong leadership
Fig. 4: Total estimates of outreach by HOPES site.
Assessing early impacts of community collaborations can be challenging, as many coalitions undergo a period of initial developmental “growing pains” that require time, energy, and effort to set the foundation for a successful collaborative. Based on estimated data submitted by HOPES program sites, over 1 million individuals (may include duplicate counts of individuals) were reached during coalition activities and outreach conducted as part of HOPES programming from 2015 – 2018.
While each HOPES collaboration was different in scope and goals, the need for consistent communication and strong leadership were the top themes gathered from our research from surveying and talking to stakeholders.
Check out our full report for more information!
| Thank you to all the staff and clients who welcomed us into their work and their homes. We are humbled by your dedication and commitment to improving the lives of Texas children. |