Benchmarking client satisfaction with general practitioner services: results and lessons from a population based survey. Ciaran O’Neill
Reader in Health Economics and Health Policy School of Public Policy, Economics & Law, University of Ulster at Jordanstown, Shore Road, Newtownabbey Co. Antrim, BT37 0QB External Fellow, Health Economics Unit, School of Economics, University of Nottingham.	Summary / Introduction /Method / Results / Conclusions / Table 1 / Table 2 / Appendix 1 / Appendix 2  References

Client satisfaction is an issue that in the interests of both the client and the practitioner, GPs should have regard to. In this paper a population based survey, accessible to GPs, is analysed to shed light on the issue of satisfaction. Several factors significantly correlated with satisfaction are identified. The value of such analyses in identifying factors under the GPs control to enhance satisfaction is discussed. The importance of controlling for other factors not under the GPs control when comparing a practice’s own client satisfaction with that of others is highlighted. The existence of similar data sets held in other countries is referred to. 

^Introduction

Various studies have shown client satisfaction to be positively correlated with clinical outcomes and with service utilisation^1-5. These are findings consistent with intuition, individuals whose experience of a service is positive being more likely to comply with the treatment recommended by its provider^6-8, as well as to seek treatment in a timely fashion^9,10. (By the same token individuals who have found a service to be effective are more likely to have higher levels of satisfaction with it.) Similarly, that dissatisfaction may deter use or see patients defect to other providers, is consistent with expectations. It, follows from these results that both in their own interests and those of the patient, practitioners may wish to monitor the degree of satisfaction experienced by clients (and do so against that of other practitioners) as well as identify the factors that influence this.

A number of studies have examined satisfaction specifically in a general practice context^r8,11-18. These have focused on issues of  methodology^16-18,  the impact of specific services on satisfaction^11-14 and the importance of satisfaction in marketing activities¹⁵. While shedding light on these issues,  the studies are not particularly helpful in facilitating individual practitioners to benchmark their services against those of others i.e. to assess their relative performance. In assessing this or in signalling it to clients or policy makers, it follows that they are of limited value. In this paper I examine the potential of archived population based data to serve in this regard as well as to inform the practitioner on improvements to service delivery that will enhance satisfaction.

Method

To investigate this issue reference was made to the Northern Ireland Social Attitudes Survey¹⁹. This is an annual survey of Northern Ireland households that examines population attitudes to a range of  issues including client satisfaction with GP services. (Fuller details of the survey, the sampling procedure used etc are presented in Appendix 1 and full details discussed elsewhere¹⁹.) This data was used simply because of its accessibility to the author. An internet search revealed the existence of similar data sets in Great Britain (available through the Data Archive at the University of Essex),  Australia (available through the Social Science Data Archives) and the US (available through the  Inter University Consortium for Political and Social Research). It follows that the approach adopted here should be repeatable in other contexts.

 Whether directly from the survey or other sources in the public domain it was possible to locate data on respondent satisfaction with GP services together with a comprehensive list of variables likely to explain this. Data extracted from the survey may be gathered under four headings. First, satisfaction with service. This was measured in the survey using a five point ordinal scale, ranging from very satisfied,  satisfied,  neither satisfied/dissatisfied, and dissatisfied to very dissatisfied. Second, socio-economic data relating to the respondent and likely to influence satisfaction. This related to the respondent’s age (AGE), sex (RSEX), income (INCOME_1…INCOME₅), highest educational qualification (CSEO, ADEG, DEGREE as appropriate) and religion (CATH). The latter was included as in the context of Northern Ireland, it has been found to be related to a number of measures of deprivation²⁰ and may, therefore, serve as a proxy for these. Third, the respondent’s perceptions of GP services. Specifically, the ease with which the respondent believed s/he could change GP (GPCHANGE₁...GPCHANGE₃), the weight s/he believed would be attached their opinion in choosing a hospital should they be referred to one (WCHHOSP₁...WCHHOSP₃) and whether or not s/he believed GPs would treat members of both religious traditions in an equitable manner.  (Given Northern Ireland’s recent history, the latter variable may be viewed as a proxy for professionalism on the part of the practitioner – i.e. that s/he is unaffected by sectarianism in the performance of their job.) The rationale for including such variables should be self-evident. Finally, other data - related to household characteristics that might affect demand for services and thereby satisfaction. These included whether or not the respondent’s household had an under 5 year old, an over 75 year old or a disabled person (RISK) in it, whether the respondent was an informal carer and whether or not the respondent was covered by private medical insurance (PRIVMED). These data were extracted from the survey and imported into LIMDEP Version 6.0²¹ for analysis.

  In part the choice of variables was governed by a review of the literature in this area and in part by economic theory. As the primary purpose of the study was to ascertain the usefulness of the data set to practitioners rather than define the relationship between satisfaction and respondent characteristics this approach to function specification was not considered to be a major drawback to the study.

  Satisfaction, of course, is only partially determined by respondent characteristics. The availability of GP services in the respondent’s locale are also likely to affect reported satisfaction. From published annual accounts of health purchasers in Northern Ireland information on the number of GPs per capita (GPPC) in the respondent’s locale, together with expenditures on GP services per capita (GPEXPPC) were gathered. In Northern Ireland this data could be disaggregated to four areas, these corresponding to the four administrative areas covered by government purchasers of health care. Each of these covered a population of approximately 0.5 million persons. Examination of similar data for GB²² indicated that greater levels of disaggregation were attainable elsewhere. Full details on the definition of all variables used is presented in Appendix 2.

  The ordinal nature of the dependent variable, satisfaction, required the use of a technique known as ordered logistic regression²³ for data analysis. This technique produces estimated coefficients on dependent variables, together with estimated “thresholds”. The latter define where, given the characteristics of the respondent, it is estimated that s/he will move between one level of satisfaction and another. The effect of changing an explanatory variable (for example increasing income) upon satisfaction cannot be directly inferred from the estimated function i.e. estimated coefficients cannot be interpreted directly as marginal effects. Rather marginal effects must be estimated from the function. Nevertheless, it is possible from the function to determine the relationship between satisfaction and the variables believed to explain it. Similarly, it is possible to identify expected satisfaction with services for a given vector of characteristics and for the practitioner to compare this with the levels they achieve. In short from the estimated function, the practitioner can identify those factors within their control that can be used to enhance satisfaction as well as, for a given set of client characteristics, the satisfaction levels other practices attain. Surveying the satisfaction of his/her own client’s, using the estimated function to control for the effects of age, sex, income etc., the performance of own practice relative to that of others can be determined.

 Results

A usable sample of 980 responses were available from the survey for analysis. (Only individuals providing usable responses to all the questions specified the function were included). In Table 1 the results of the regression analysis are reported and in Table 2 the nature and significance of the relationship between satisfaction and the various explanatory variables based on log-likelihood ratios are shown. (The latter allow for groups of variables e.g. those relating to income to be tested collectively.) The relative magnitudes of c² values reported in Table 2 - the results of the likelihood ratio tests - can be used to infer the ordering of variables as determinants of satisfaction - a higher c² value denoting a more significant determinant of satisfaction.

From Table 2 it can be seen that the key determinants of satisfaction with services were, in order of importance, education, perceptions as to the weight attached respondent wishes on hospital referral, expenditure on GP services per capita, age, perceptions as to the ease with which the respondent could change GP, that the health service treated members of Northern Ireland’s two communities equally, the number of GPs per head and the respondents sex. Other variables were not significant.

From the tables it can be seen that males were less likely to be satisfied with services than females (because of the ordering of satisfaction, a positive result should be interpreted as indicating lower satisfaction). Those with positive perceptions as to the ease with which they could change GP were more satisfied than those who had not. Those who perceived themselves to have a greater input into the decision of where they might be hospitalised (should the need arise) were more satisfied than those who had not. Satisfaction was also positively correlated with expenditures per head of population on GP services. Individuals less than 65 years of age,  individuals , who believed health services were delivered differently to the two communities and individuals who were better educated were less satisfied with GP services than those who were not. Perhaps paradoxically, given the relationship between expenditures and satisfaction,  the greater the number of GPs per head of population the lower was satisfaction with GP services. Income, religion and private medical insurance were unrelated to satisfaction as was whether the respondent had an under 5 year old, over 75 or a disabled person in their household.

  Discussion

Highest educational attainment was found to be the variable most strongly related to satisfaction. Better educated individuals were less likely to be satisfied with GP services.  Intuitively, a number of plausible explanations for this result can be offered.  Higher levels of education may be associated with higher expectations of GP services or higher levels of education may result in individuals having less confidence in the ability of their GP. For whatever reason the relationship exists, it is a result that is consistent with that reported elsewhere²⁴. That this is neither a novel finding nor one that relates to a variable over which the practitioner has control could of course provide a basis for questioning its value. However, in as much as it is important to demonstrate the generality of this relationship (i.e. that it exists also in this context) and the need to control for its effect when making between practitioner comparisons in client satisfaction, it is contended it does have value. By extension the value of making similar (or dissimilar) findings in other specific contexts using similar data sets should not be underestimated. This is also the case with respect to client age and sex both mirroring results found elsewhere¹⁵ and both found to be significant or marginally significant here. In relation to client income and religion as well as to characteristics of the household, not found to be significantly related to satisfaction, the study allows the practitioner to identify this fact from data available to them and to consider ignoring such data when surveying own satisfaction. Thus again the exercise is seen to have value.

  The perceived weight given to client wishes in the choice of hospital is highlighted by  the positive correlation found between satisfaction and this variable. This no doubt reflects concerns about access for friends and family to the hospital rather than issues of technical competence, though that this is also a concern cannot be ruled out. In any event this finding identifies an area under the control of the GP where action is possible to enhance satisfaction. Thus, reassurance from the GP in respect of choice of hospital should result in enhanced client satisfaction. Given reassurance should be achievable at minimal costs, clearly it is an action practitioners should consider. This is similarly the case in respect of perceived ease of changing GP.

  Turning to expenditures on GP services in the respondent’s locale, as with client income, sex, age etc. this will be beyond the control of the individual practitioner. As with these variables that its effect should be controlled for – given in this instance its significance in the function – is nevertheless clear. Thus, this result indicates that a practitioner located in an area where expenditures are low, will ceteris paribus, experience lower satisfaction with services. Clearly, this is a finding the practitioner and those assessing his/her performance on this criterion should be aware of. This is similarly the case in relation to GPs per head of the population in the respondent’s locale though the marginal significance of the variable – (seen in Table 2 to be significant only at a = 0.1) suggests that the case for control is less strong.

  Finally, here it is perhaps noteworthy that while respondent satisfaction was positively related to expenditure it was negatively related to the actual numbers of GPs per capita in the respondent’s area. At first glance this result may seem paradoxical given the positive relationship between satisfaction and expenditure. However, a possible explanation may be the role of expectations in determining satisfaction. Thus as the number of GPs per head of population rises so too does the pool from which the respondent can choose to receive treatment (in economic terms the range of available substitutes increases). As the range of substitutes increases so the respondent may become more demanding of any one GP.

Conclusion

The study has provided four findings worthy of note. First, there exists data to which GPs can gain access that contains information on satisfaction with GP services. Second, these data can be used to identify factors over which the GP may have little control but that may nevertheless affect satisfaction with his/her services. When comparing client satisfaction between practitioners, when reporting this information to clients or policy makers and when considering the need to respond to dissatisfaction among clients it is important that the practitioner be aware of these. Third, these data can be used to identify the impact on satisfaction of factors under the GP’s control. From this the GP can devise strategies to enhance the satisfaction of their clients. Finally, estimating the function within the practitioners own context – Northern Ireland, GB, the US etc. - will produce results more meaningful to practitioners and - if they have surveyed their clients’ satisfaction – provide results permitting comparison of their performance with that of others with whom they may compete. 

  Table 1.

Results of Ordered Logistic Regression of Respondent Satisfaction on

Explanatory Variables (very satisfied = 0, satisfied = 1,

neither satisfied/dissatisfied = 2, dissatisfied = 3, very dissatisfied = 4)

N= 980

Table 2.

Collective Significance of Variables

* indicates significant at 90% level of confidence, ** significant at 95% level of confidence and *** significant at 99% level of confidence. Wald statistic based on the difference in the Log-likelihood values of the function with and without a variable present in the estimated function.

Appendix 1.

The NISA survey is based on a stratified random sample of households drawn from the local government rating lists. In 1994/95, the year for which the current study draws data, 2400 households were selected for interview of which 233 proved to be vacant properties. Of the remaining 2167 contacted, interviews were achieved with respect to 1519 though not all households were surveyed with respect to each aspect of the questionnaire. Comparisons of responses to the NISA survey with those to much larger surveys of the NI population such as the Continuous Household Survey and the Northern Ireland Census suggest that there is no reason to believe non-response bias with respect to socio-economic, age or religious grouping was evident in the sample. This can be seen, for example, by reference to the table below and would suggest it is representative of the Northern Ireland population.

Sample characteristics against those found in larger surveys