Community-based participatory research (CBPR) aims to promote community-level change through integrating the rigor of academic research with the knowledge and experience of community partners.1-3 By involving those directly affected by the issues under study, CBPR ensures local relevance that can foster lasting social change. Because of this inclusiveness, CBPR has been credited to be a promising approach for the reduction of health disparities and as an effective way to create sustainable community outcomes.2,4-6 A diverse array of community partners creates opportunity for numerous community issues to arise. However, a diverse array of community partners also creates the need for strategized prioritization, an important step toward community coalition building.7 Priority setting is an essential decision-making step in community-based participatory research. However, there is little guidance on how to approach priority setting with objective measures while implementing CBPR. A more robust way to set priorities is by applying consensus building methods, such as the Delphi technique,8-14 and nominal group technique (NGT).15,16 These techniques are well-established procedures used to gather consensus across diverse groups of stakeholders.9-11,13,14,17 Their common denominator is built-in ranking procedures and consensus. The Delphi technique encompasses several iterations of ranking and re-ranking (typically three or four) and consensus discussions; whereas the nominal group encompasses only one rank-ordered feedback and discussion. The Delphi technique has strength in its replication, but it can span several weeks and often requires the technical skills of academics, highly skilled facilitators.8-10,13,14 Because nominal groups are easier to be implemented, they tend to be popular and favored in community settings. The underlying problem with this simplistic approach is that in reality multiple criteria are at play and individual decisions are complex. When asked to provide rankings based on their own informed opinion, community participants may apply a different implicit criterion in their ranking. Some decision-makers may decide on a priority issue based on the known burden of disease, another group may do so based on their desire to reduce health inequalities of disadvantaged groups in the community, whereas others may do so based on practical and budgetary constraints. Unless decision-making criteria are made explicit in the ranking process, participants may not be able to make a distinction between their own preference, community relevance, fairness, or changeability. This is particularly important in CBPR projects where multiple social-ecological determinants of health require attention, but projects operate under relatively short-timeframes and small budgetary resources.
The adoption of a multi-criteria decision-making approach in health priority setting has been widely recommended, but there has been little integration in the healthcare arena and even less so in community programming settings.18-21 Decision-makers often set priorities based on unreliable subjective criteria and the allocation of resources is therefore, sub-optimal. This situation occurs because decision-makers, if unguided, will find it very challenging to condense rationally the multiple criteria at hand into a single vote. Therefore, the process becomes unreliable and irrelevant; thus, the need for a systematic, intuitive, and transparent priority setting in CBPR.
Several techniques have been proposed to combine multiple decision-making criteria, including qualitative analysis of performance matrix, preference-based surveys (population-based), simple linear additive evaluation models, conjoint analysis, among others.20,22 Due to the complexity and the high analytical skillsets required, these techniques are often out of the reach for community-based organizations with scarce resources. Providing practitioners with a simple and intuitive measure for priority setting that can be combined with diverse data collection methods is a necessary step to foster accountability of the decision-making process in community settings. Yet, there is a lack of easy-to-use, but methodologically robust measures, that can be feasibly implemented for reliable decision-making in community settings. To address this important gap in CBPR, we demonstrate the applicability of an index previously developed by our team under the framework of CBPR. This paper described in detail the testing and validation of a community priority index.
Design and Methods
This study was conducted as part of a CBPR project titled Toward Eliminating Disparities in Maternal and Child Health Populations implemented in East and Central Tampa, Florida, USA, and funded by a National Institutes on Minority Health and Health Disparities (NIMHD) R24 grant. The main goal of the project is to identify key maternal and child health issues that need to be prioritized and addressed through a concerted community action plan. Specifically, the aim is to capture the community perspective with regards to salient women’s health issues, maternal health issues (during pregnancy), and issues for children and adolescents. The project started with a strong community leadership plan devised by a Community Advisory Board (CAB), in partnership with a non-profit organization called REACHUP Inc. and partnering researchers from the University of South Florida. The CAB is comprised of trusted leaders in Central and East Tampa with a long history of participation in community development efforts, including: 6 community members (concerned citizens), 5 community-based organization representatives, and 2 university CBPR advisors. The CAB members represent the racial/ethnic admixture of the target community, and are well-connected with local organizations, and have had experience in previous participatory research efforts. The CAB members provided guidance to the project during bi-weekly or monthly face-to-face meetings, and were directly involved in the implementation of data collection and analysis of results. University researchers were scientific advisors to the project offering recommendations that tallied with their expertise. The larger CBPR study received ethical oversight from the Institutional Review Board from the University of South Florida.
A sequential mixed-method design was used, which combined focus groups findings, nominal group (NG) technique, and the generation of quantitative scores for priority setting through the creation of a Community Priority Index (CPI). Qualitative data was analyzed through thematic coding. Quantitative data were analyzed using SAS v9.4.
To identify salient issues from the perspective of community members, a series of 10 focus groups was conducted in diverse neighborhood locations. The main question to elicit discussion about maternal and child health issues was: What makes it difficult for women to be at their best health in this community? The same question was repeated with regards to women during pregnancy, and for children and adolescents. A total of 78 community residents (8 participants per group, men and women) participated in the focus group discussions, which included 2 adolescent focus groups and 2 Spanish-speaking focus groups. Focus groups transcripts were analyzed thematically by members of the Community Advisory Board, which resulted in the identification of a list of issues relevant for women’s health, during pregnancy, as well as for children and adolescents. The summary of focus group findings was presented in 2 Community Forums for member check validation, conducted in July 2014. After compiling a list of 30 issues, there was a need to devise a method for prioritizing and reducing the list not only based on importance, but also based on relevance to maternal and child health context (by population), as well as on the feasibility of implementing change (changeability) with current resources and project timeline.
Modified nominal group technique
The prioritization process was conducted by a group of six community advisory board members (sub-committee, 6 concerned citizens) with varying levels of experience in community grass-roots development. For this purpose, we adapted the nominal group technique for the application of multi-criteria ranking by importance, changeability, and sub-population. Since the recommended composition and size of the NGT panel is between 5 to 10 people with expertise on a particular topic, this number was considered adequate.11 Members from the community-based organization and university advisors excluded themselves for the exercise to ensure only direct community representatives were taken into account. The NGT sub-study was conducted during a 3-hour meeting in March, 2014.
The first step in traditional NGT is brainstorming of issue ideas by panelists.23 However, instead of expert-driven, we capitalized on the community-driven focus group findings. Participatory focus groups are considered more credible, transferable, dependable, and confirmable than expert opinions.24 We formatted the list of issues identified in the focus groups as survey items for subsequent ranking. Next, the traditional NGT steps of ranking and consensus discussion were applied. Participants first assigned rankings to each issue separately using 2 criteria: importance (i.e. how relevant the issue was to the community context?) and changeability (i.e. how easily the issue could be changed in the community if a community health promotion program would be made available?). For this purpose, we used a 3-point Likert type scale (for importance: 0=not important, 2=intermediate importance; 3=very important; for changeability: 0=not changeable, 2=intermediate changeability, 3=highly changeable). After individual rankings, the scores for all items were summated by importance as well as by changeability. This resulted in group-based sum of item scores, rather than individual rankings. Notably, the application of a dual criteria at this stage permitted that participants decomposed their personal opinions or hidden preferences by justifiable and defensible criteria. In traditional NGT studies, only one type of raw scores is generated with no attempt to add defensible criteria in a consistent manner.
Triangulation of multiple sources of data
Focus groups and nominal groups could be a challenging and daunting task for community members if multiple criteria are to be applied. Thus, a simple, yet rigorous approach was needed to summarize findings to facilitate issue prioritization. Particularly, we needed to combine importance and changeability criteria to yield a single measure in order to identify those priorities that were both highly important from the community perspective and highly changeable. Highly important and highly changeable issues were regarded as the top priorities that must be addressed in the project context. Furthermore, priorities needed to be determined by maternal and child population (women in general, pregnant mothers, and children). The details of the development of the Community Priority Index are presented below, along with the application, and its evaluation.
Community Priority Index
Motivated by the need for combining importance and changeability and to have priorities by maternal and child health populations (focus of the grant), we demonstrated the utility, applicability, and validation of the CPI. The CPI runs under the assumption that addressing highly important topics that are very difficult to change or those highly changeable but of relatively low importance would result in sub-optimal use of the finite resources allocated to the project. The generation and computation of CPI scores was conducted in a stepwise fashion that is easy to replicate, as follows: i) multiplicative combination of the observed importance and changeability averages for each issue. This step involved the conversion of all sum of item scores into weighted scores and then into summary statistic that we refer here as CPI. Because a few items were left blank by some respondents, as often occurs in community settings, using the simple sum of item scores was inappropriate because of the unequal number of participants’ responses per item (e.g. some items were responded by all 6 members, while a few were responded by only 5 members). Thus, we used the arithmetic mean or average and computed mean importance scores and mean changeability scores. Accordingly, each sum of item scores was weighted (divided) by the number of respondents for the particular item, which resulted in the item mean importance as well as item mean changeability. Forced responses were not enforced to maintain democratic and honest responses among CAB members. Once we had mean importance and mean changeability, we then multiplied these values based on the following formula, for each issue: CPI = Mean Importance × Mean Changeability. Multiplication, rather than addition, was considered more appropriate given the synergistic nature of importance and changeability on issue prioritization. As a result, a single summary index was computed for each issue which integrated both perceived importance and perceived changeability, with higher values indicating higher priority. ii) Disaggregation of CPI scores by target population to identify priorities for action. As a final step, the issues were ordered in a descending fashion to identify the top highly important and highly changeable issues by target population. With this information, the CAB was able to clearly and democratically identify the top priorities to be addressed for 3 diverse populations separately: women outside pregnancy, during pregnancy, and children and adolescents.
Evaluation of the applicability of Community Priority Index scores and standardization
As a means to evaluate the precision of asymptotic approximations in the small sample and the potential applicability of the CPI scores, we conducted a bootstrapping data simulation experiment and applied standardization techniques. Since we had only 6 original study participants, the traditional confidence interval estimator that is based on the normal assumption of the sampling distribution cannot be used.25 To overcome this, we complemented the classic analysis with bootstrap methods to construct 95% confidence intervals.26 In bootstrapping, data collected for a single experiment is used to simulate what the results would have been if the experiment was repeated over and over with new samples. These bootstrap samples were created by sampling with replacement from the original dataset. The method provides an alternative to large sample techniques when asymptotic properties are not met. Specifically, we used bootstrap samples to estimate the mean score of 3-points Likert type scaled items and their 95% confidence interval. By using bootstrap methods the distribution of the data normalizes permitting the use of the mean as a reference cut point.26 Therefore, to improve the precision of the estimates generated by a small expert panel (CAB members) in the assessment of perceived importance and changeability, we generated via computer program (S+ 8.2), 5000 bootstrap samples of community board member ratings.27 We used the following algorithm to generate the bootstrap samples:
we constructed an empirical distribution function, F̂, from the observed data. F̂ places probability 1/n on each observed data point x1, x2, …, xn (x=6);
we then drew a bootstrap sample X1*,X2*…,Xn* of size 6 with replacement from F̂. The mean of this bootstrap sample was calculated achieving a normally distributed population;
step 2 was repeated 5000 times. The percentile method was used to compute a 95% confidence interval around the mean by ranking the bootstrap sample means, and then selecting the 2.5 percentile as the lower confidence limit and the 97.5 percentile as the upper confidence limit.
Since our results were scale-dependent and lacked comparability potential with other studies, we decided to standardize each CPI indicator to have a range from 0 to 1 by applying the following formula:
The lower bound value is the least score possible within the 95% confidence interval, while the upper bound value is the highest possible score within the interval. Given the above formula, the CPI can only range from 0 to 1 and it’s scale-free, which now permits comparisons across different studies and populations. In this manner, CPI scores provided a roadmap for prioritization, where the mean indicates the group consensus and the width of 95% confidence intervals indicate the range of agreement. The entire process of the generation of the CPI values is depicted in Figure 1.
Applicability: top community priorities using Community Priority Index
Next, we applied the derived CPI scores to identify the top health priorities for community action to eliminate disparities in maternal and child health populations. We adopted the following cut-off points for prioritization based on importance and changeability: values <0.3 were considered low priority, 0.3-0.7 intermediate, and >0.7 high priority. Table 1 presents the obtained importance and changeability values, as well as the computed CPI values in the original units by maternal and child health populations.
Subsequently, the obtained CPI values were transformed so that the scores would range between 0 and 1, so that the CPI values were not depending on the scales used and were easier to interpret. Table 2 presents the standardized values.
For women’s health in general, top priorities were: low health literacy (CPI=0.87; 95%CI: 0.69, 1.00), low educational attainment (CPI=0.78; 95%CI: 0.48, 1.00), and lack of self-esteem (CPI=0.72; 95%CI: 0.44, 1.00). The lowest priorities were: lack of spirituality, violence in the community, and grandmothers raising grandchildren.
For children and adolescents, the top three priorities were: obesity (CPI=0.88; 95%CI: 0.69, 1.00), low self-esteem (CPI=0.81; 95%CI: 0.69, 0.94), and negative attitudes toward education (CPI=0.75; 95%CI: 0.50, 0.94). The lowest priorities were: inadequate parental monitoring, issues with fathers or males in the community, and overuse of technology.
After the ranking process, a discussion followed on how top priorities were issues that affect very negatively the quality of life in the community and how CAB members felt confident that the CBPR project could address those through community efforts. With regards to the lowest priorities, CAB members indicated such issues were somewhat relevant for the community, but were deemed as having low changeability (i.e. single parenting). Some members also indicated that some low priorities impacted fewer families in the neighborhoods (e.g. medicalization of pregnancy), compared to the top ones (e.g. stress, obesity). This permitted a clear identification of concurrent priority issues across populations, which were issues related to emotional health (i.e. maternal stress, and lack of affection, low self-esteem) and nutrition (i.e. both for mothers and for children). Low health literacy also emerged among the top concerns for women. Prior research has indicated that many women are interested in improving these priority areas and their overall health.28 Health literacy is further complicated by the need for ehealth literacy, as the work becomes a more globally connected place.29 Because of women’s health directly influence pregnancy outcomes and the important role of maternal health on the health of children, the CAB members set the objective of improving emotional health and nutrition for women of reproductive age in the community. This decision received positive reactions through community forums conducted in the community in subsequent months. Thus, the priority setting using the CPI achieved the expected objectives of helping decision-makers in the separation and application of multiple decision-making criteria and selection of priorities for action.
Evaluation through data simulation
In Table 2, it can also be appreciated that the 95%CIs provide an opportunity to assess the precision of the CPI scores. Particularly, the CPI score indicate the level of group consensus on importance and changeability, whereas the width of the 95% CI indicates the scope of agreement. For example, for women during pregnancy (Table 2), stress was selected as the top priority as indicated by the highest CPI score closer to 1 (CPI=0.85; 95%CI: 0.70, 1.00). Of all three, the 95%CI was also narrower, indicating a high level of group agreement for this issue. Lack of affection (both emotional and physical affectivity) was also a top priority as indicated by CPI score of 0.87. However, the 95%CI was slightly less precise. Nutritional issues were also very important and very changeable, as indicated by a CPI of 0.78. Of the top three priorities, nutritional issues presented the wider 95%CI, which indicates that group opinions were more disperse. Using our recommended cutoff values, practitioners can choose not only priorities that were highly important and highly changeable, but also select those for which there is greater agreement.
In summary, our study found that applying a CPI can be a useful tool in CBPR. Although this instrument was created by our team in the United States, it has the potential to be used in other contexts and countries. In fact, the duality of an index that addresses both changeability and importance may be well suited for application in varied geographical contexts, as it takes into account the natural effect culture has on prioritization.
This study describes the use of a community-driven measure of priority setting, namely, the Community Priority Index (CPI). This index was created based on multiple criteria: importance and changeability, as well as stratified by target population. Importance and changeability are 2 commonly used criteria in planning frameworks.30,31 Yet, there is no single measure or quantification process that ensures objective decision-making, while incorporating aspects of relevance and changeability. By integrating well-established frameworks with rigorous measurements for priority setting lends credibility to the Community Priority Index.
Our findings suggest that a CPI approach to prioritization yields an intuitive measure based on community concerns. CPI represents not only an applied measure, but also preserves the rigor and objectivity of standard priority setting measures. Community-based organizations can utilize this systematic CPI approach in a flexible manner to complement both qualitative and quantitative needs assessment methods. Despite a small sample size, this study demonstrates how by applying a simple index, CBPR can provide a reliable, credible, and applicable information for decision-making. Furthermore, the CPI can be incorporated as a component of more advanced and complex program planning efforts, such as PRECEDE-PROCEED, Intervention Mapping, and Planned Action Toward Community Health.30,32 We argue that issue prioritization must be driven not just by the importance of the issue, but also what realistically can be changed/realized with available funds. In summary, the CPI is shown in this study to be a viable approach to address the gap in objective measures for community-based decision-making.