INTRODUCTION

Top Abstract Introduction Methods Results Discussion References Appendix I Appendix II Appendix III

COMPLEX PHYSIOLOGICAL responses, such as glucose tolerance, may be assessed by many different methods. For instance, the control of plasma glucose may be measured by the fasting plasma glucose (FPG), by the response to an oral or an intravenous glucose challenge, by the percentage of glycated hemoglobin (HbA_1c), or by the plasma concentration of fructosamine. Different clinical tests are used to assess the same physiological variable because of different clinical circumstances, the application of advances in understanding and technique, or as a result of historical circumstances or fashion. The tests may assess differing or equivalent aspects of a physiological characteristic and may express their results in different units of measurement.

In this paper, we recommend a simple but comprehensive methodology for comparing different tests of a continuous physiological variable, which may be applied irrespective of the scales of measurements used. For such comparisons, it is important to include assessment of both the between- and within-subject variation of each test, as this allows comparison of the ability of tests to discriminate between different subjects, determination of the underlying correlation between tests after adjusting for attenuation due to within-subject variation, and unbiased estimation of the underlying relationships between the results of the different tests.

Omission of any of these components in a comparison study will provide inadequate information for choosing a particular test for a particular situation. The imprecision, or the within-subject variation, of a test is of little use by itself and must be considered in relation to the range of the test results. The "coefficient of variation," which relates imprecision to the midpoint of the range, is often inappropriate as it does not take account of the dynamic range of a test (1) and is not always comparable between different scales of measurement. The present paper introduces the concept of "discrimination," i.e., the ability to distinguish between individual subjects within a specified range of interest. This can be expressed as the discriminant ratio (DR), which is defined here as the ratio of the underlying between-subject SD (SD_B) to the within-subject SD (SD_W). The DR has a defined distribution, and DRs for different tests can be compared statistically.

The correlation between different tests must be included in any comparison. Test imprecision will diminish, or "attenuate," the measured correlation coefficients in a well-described way (11), and it is important to correct for this so as to be able to assess the underlying "true" correlation, as this represents the degree to which the tests are assessing the same physiological trait. This attenuation adjustment can be expressed in terms of the DRs of the respective tests.

Finally, the comparison of tests must, if possible, relate measurements by one test to those by another. Where the relationship is linear, the gradient of the relationship obtained by least squares regression is underestimated ("regression dilution") when both tests are subject to appreciable measurement error. An unbiased technique for deriving the relationship is therefore necessary, and, while many methods are available, a suitable method (27) is recommended here.

The assessment of glucose tolerance is a specific area where several methods are available to an investigator, for instance, by the measurement of steady-state plasma glucose or the response to a glucose challenge, either oral or intravenous. The standard test of glucose tolerance has, until recently, been the oral glucose tolerance test (OGTT; see Ref. 33), but, in practice, this test is not often performed. This is partly because of the inconvenience of a 2-h test and partly because of the marked variability of the OGTT. The poor reproducibility of the test (12, 21, 24, 26), which has a reported coefficient of variation of the order of 15-40%, due in part to the variable rate of gastric emptying, has predictable effects on reclassification of subjects on repeat tests, with a change in status on repeat testing in 30-60% of cases of impaired glucose tolerance (IGT; see Refs. 9, 13, 26, 28, 29) and predictable regression to the mean (26). The simple measurement of FPG has been suggested as a preferable measure (4, 8, 15, 20), and a continuous intravenous infusion of glucose (CIG) can assess glucose tolerance and give simultaneous measures of pancreatic -cell function and insulin resistance (16). The FPG thresholds for diabetes have been set using outcome data (4). We evaluate and compare the performance of this and other tests throughout the physiological range.

This paper outlines the concepts and components of a physiological comparison study and uses as an example the assessment of glucose tolerance by either 1) the FPG (single sample or mean of 3 consecutive samples), 2) the 1- and 2-h responses to a CIG, and 3) the standard 2-h response to the OGTT, in repeated tests in 30 subjects spanning the range of glucose tolerance.

	METHODS

Top Abstract Introduction Methods Results Discussion References Appendix I Appendix II Appendix III

Statistical Methods

Each of these aspects is required for a comprehensive comparison study and is based on a combination of well-recognized and novel concepts. Our approach is considered on a conceptual level at this point, with a more rigorous statistical treatment reserved for APPENDIXES I-III.

Discrimination between subjects. All physiological measurements 1) are subject to imprecision, which may derive from biological, sampling, and analytic sources and 2) relate implicitly to variables taking on values within a particular range of interest. The performance of a particular physiological test will depend on the relationship between both of these characteristics. Absolute measurements of the imprecision of the test are only meaningful in relation to the range of values to which that test will be applied. The smaller the former is in relation to the latter, the greater is the ability of a test to discriminate between individual subjects. In the context of measurements being obtained from a series of individuals representing the physiological spectrum of interest, we propose a novel, simple index of discrimination, the DR, the ratio between the SD of the underlying subject means (SD_U), and the SD of repeated measurements on the same subject (SD_W).

The discrimination of a test is not a universal property but will relate to the spectrum of values in the population being studied. Hence absolute DR values are not comparable between different populations, but they are essential when comparing the practical application and performance of different tests in the same population.

Underlying SD_U. Because a physiological test may be applied to a variety of possibly nonuniform populations, it is important to assess the test in relation to its expected range of application. Subjects in a comparison study should be chosen to represent and to span the range rather than be randomly selected from particular populations of interest, and this range is characterized statistically by the SD_U. The measured SD (SD_B) will overestimate the underlying SD_U due to the presence of within-subject variation, and it is important to adjust for this, using a standard formula, to yield an unbiased estimate of the SD_U.

SD_W. To relate simply to the between-subject variation, we must be able to assume a common within-subject variation for all of the subjects in the study. This property is called "homoscedasticity" and can be checked by simple plots of the data (see APPENDIX I). Lack of homoscedasticity can often be rectified by an appropriate numerical transformation of the test results.

Experimental Protocols

Protocols. Each subject was studied on four occasions within a 6-wk period. After a 12-h fast, subjects went to the hospital and sat on a bed for the duration of the tests. Tests were each performed on two occasions in the same subject and in random order.

FPG and CIG. Two cannulas were inserted in the same arm. One, for blood sampling, was placed at the wrist or on the dorsum of the hand, which was heated by an electrical blanket to "arterialize" the venous blood. The other cannula, for infusion of glucose, was placed in an antecubital vein. A blood sample was taken at time 10 min, and the plasma glucose concentration at this single time point was termed FPG₁. Blood samples were also taken at times 5 and 0 min, and the mean of the plasma glucose at the three time points was termed FPG₃. At time 0, a continuous 5 mg · kg ideal body wt¹ · min¹ infusion (22) of 10% glucose was started and continued for 2 h. One-hour CIG and 2-h CIG glucose were defined as the means of the three plasma glucose concentrations in blood samples taken at 50, 55, and 60 min and at 110, 115, 120 min, respectively.

OGTT. A single cannula was placed in an antecubital vein for blood sampling. Fasting blood samples were taken at 10, 5, and 0 min. At 0 min, the subject consumed a 75-g glucose drink, and blood samples were taken at 30, 60, 90, and 120 min.

Biochemical Assays

	RESULTS

Top Abstract Introduction Methods Results Discussion References Appendix I Appendix II Appendix III

Estimates of Glucose Tolerance

View larger version (10K): [in this window] [in a new window]   Fig. 1.   First minus second test difference vs. mean plots for the mean of three 5-min samples of fasting plasma glucose (A) and 1-h (B) and 2-h (C) plasma glucose during a constant 5 mg · kg ideal body wt1 · min1 intravenous glucose infusion (CIG).

View larger version (11K): [in this window] [in a new window]   Fig. 2.   First minus second test difference vs. mean plots for 2-h oral glucose tolerance test (OGTT) for untransformed plasma glucose (A) and glucose logarithmically transformed to ensure homoscedasticity (B).

DR values for the five measures, together with the one SE range of the estimates, are illustrated in Fig. 3. Although the lowest DR was the FPG₁ and the highest the 2-h CIG, there was no significant difference between them on the overall statistical test (²₄ = 6.2, P = 0.19, using Eq. 10 in APPENDIX I). Consideration was given to the exclusion of a subject whose inter-test difference in the 2-h CIG was four SDs from the mean of the rest of the group (see Fig. 1). In the absence of an identifiable reason for the large difference between his two test values, this subject was included in the analyses presented here, although, if he were excluded, the DR of the 2-h CIG would rise to 6.1, significantly greater than the DRs of the other tests.

View larger version (9K): [in this window] [in a new window]   Fig. 3.   Discriminant ratio values for fasting plasma glucose (single sample and mean of 3 samples), 1-h CIG, 2-h CIG, and 2-h OGTT for plasma glucose. Error bars represent 1 SE of the estimate.

Table 3 shows the Pearson correlations between the tests, both before and after adjustment for attenuation. The correlations were calculated between subject means of duplicate tests, to give the best estimate of the underlying relationships between tests. Adjusted correlations between the fasting and both of the intravenous measures were high, approaching one. Those between the 2-h OGTT results and the other tests were somewhat lower (~0.9), indicating that there was some biological discordance in their relationship, independent of their within-subject measurement error.

Figure 4 shows the scattergram between the 2-h plasma OGTT (on a logarithmic scale) and the FPG. It also illustrates the line of equivalence, derived as explained above, and the linear regression line of the 2-h OGTT on FPG. The dilution effect of the within-subject variation on the regression line can be seen clearly. Table 4 gives coefficients for the unbiased linear equations relating the test values, and Table 5 gives the points on the various scales that are equivalent to the 1985 WHO OGTT thresholds for IGT and diabetes.

View larger version (12K): [in this window] [in a new window]   Fig. 4.   Scattergram of the mean of two 2-h OGTT tests (logarithmic scale) vs. the mean of two FPG3 tests showing the unbiased linear relationship (dotted line) and the linear regression relationship (dashed line).

SD_W of the logarithm of the 2-h OGTT may be interpreted in relation to a standard interval in the range of glucose tolerance, for instance, the interval between the thresholds for IGT (7.8 mmol/l) and for diabetes (11.1 mmol/l). This interval is 0.153 on a logarithmic scale (base 10), whereas the SD_W is 0.060. The difference between two individual measurements at either end of this interval would not be significant at the 5% level (P = 0.07), given the imprecision of the 2-h OGTT. In other words, it would not be possible to confidently distinguish between two such individual values. The same applies to all other tests examined, using the unbiased equivalent values to these thresholds given in Table 5, apart from the 2-h CIG plasma glucose, for which the two measurements at opposite ends of the equivalent interval would differ significantly (P = 0.013).

	DISCUSSION

Top Abstract Introduction Methods Results Discussion References Appendix I Appendix II Appendix III

This study has shown that different methods of assessing glucose tolerance were broadly comparable in a range of subjects spanning normal glucose tolerance, IGT, and type II diabetes. This assessment involved the following three separate components: 1) comparison of the discrimination of the tests, i.e., their ability to distinguish between different subjects, 2) determination of the degree to which different tests measure the same underlying physiological property, and 3) estimation of the underlying relationship between the test results. The assessment of the within-subject imprecision of each test is a fundamental requirement for this evaluation, so that comparison studies must involve at least duplicate measurements in all subjects in order to determine both the between- and within-subject measurement variation of each test in the same group of individuals. This is best done in subjects who represent a clinically meaningful range of glucose tolerance. To ascribe a single value to within-subject imprecision requires homoscedasticity, and numerical transformation of results may be necessary to achieve this, as illustrated by the 2-h plasma glucose from the OGTT. A measure of imprecision is important for the assessment of changes within subjects, such as over time or after interventions (10).

The determination of imprecision alone is not adequate, however, for the assessment of the practical value of a test. The standard methods of assessing imprecision, including the coefficient of variation, have little meaning on their own, without reference to the range of measurements to which they are being applied. The ability to distinguish between individuals within this range is here termed the discrimination of the test and is assessed using the DR.

The concept of discrimination should be distinguished from the ability of a test to categorize patients by an external gold-standard dichotomy. This is a particular concern in the field of clinical chemistry, for instance, when a biochemical test is being assessed for its ability to detect the presence of a malignancy. Receiver operating characteristics (34) have been used for this purpose. However, they are not suitable for the assessment of test results as a continuous scale of measurement or for the comparison of tests without reference to an external categorization. In the field of glucose tolerance, categories have been defined on the basis of thresholds in a continuous scale of measurement for the OGTT based on external criteria, but this is a notoriously imprecise test (7), and it would not therefore be appropriate to assess possible alternative tests using a categorical approach based on these thresholds. The use of the DR provides a means of comparing how well the subjects studied can be reliably distinguished by different tests, which is an important component of a comprehensive comparison of imprecise tests. For instance, in many research studies using continuous variables, the statistical power to distinguish between groups of subjects or to determine correlations between variables will depend on discrimination.

A very similar concept, referred to as reliability, has been assessed, particularly in the psychological literature, using the ICC. This relates the covariance of replicate test results to the combined between-subject and within-subject variance and is algebraically related to the DR. However, in the context of discriminating between different subjects, it is not as easy to grasp as the DR, which has a direct intuitive relevance when considering a test's application. Furthermore, it is not as easy to derive statistical tests comparing different ICC values. A recently described method for the comparison of two ICCs does not extend to the comparison of more than two tests and was only validated for studies employing repeated tests in 100 subjects or more (2).

The choice of a methodology for comparing different tests is intimately related to its theoretical basis and in particular to the availability of statistical criteria for assessing such a comparison. Much of the theoretical discussion of the ICC and treatments of comparative tests between ICCs have been based on random effects models in which the individuals in which the tests are being performed are assumed to be drawn at random from a known normally distributed population. This presupposes a particular population in which the tests are being applied. Complex populations (for instance those consisting of subgroups) would require appropriately complex statistical models. Unfortunately, even the simplest random effects models are hard to treat analytically, as in the case of the ICC quoted above. Our approach has been to concentrate on evaluating tests across a particular physiological spectrum of interest. In the example presented here, for instance, glucose tolerance represents a physiological and pathological unity irrespective of the its distribution in particular populations. For this purpose, we take the view that it is appropriate to perform a comparison study in subjects selected to span the range of interest, which may be analyzed by a fixed-effects statistical model. Such an analysis is presented here for the DR, allowing the derivation of straightforward expressions for the SD and confidence intervals of the DR and the evaluation of the statistical significance of differences between the DRs of different tests.

Although the comparison of the DRs of different tests in the same study is valid, the DR in itself is not a universal characteristic of a test, as it will depend on the choice of subjects on which the comparison is performed. When the subjects cover a greater range of values, the DR will be larger, and vice versa, and the DR calculated when subjects are selected to span a range of interest will generally be larger than if subjects had been chosen randomly from the same population. However, it is a fundamental property when it comes to a test's practical application, and, unlike imprecision, it can be used as a basis for comparison between tests. The DRs of the five tests examined here were not significantly different, in spite of the increased complexity and expense of the CIG and the OGTT.

Two perfectly precise tests assessing the same physiological variable would be perfectly correlated. Departures from perfect correlation can be due to the following two factors: 1) underlying differences not directly related to the variable of interest. These will manifest themselves as systematic differences between subjects; for instance, in the assessment of glycemic control, which is determined by the FPG and OGTT in qualitatively different ways, the underlying correlation may fall short of unity because of the influence of factors such as the effect of gastric emptying and the influence of intestinal incretin effects that differ between the two methodological approaches; 2) diminution of the underlying correlation may also arise from within-subject variation, and this is a well-described statistical effect termed "attenuation," which may be adjusted for by standard techniques to enable the estimation of the underlying correlation (11). This adjustment depends on both test imprecision and the degree of variation between subjects and so can be expressed in terms of the test DRs.

Adjustment for attenuation will establish the degree to which the underlying correlation differs from unity due to the factors detailed in factor 1 above.

In this paper, the adjusted correlation coefficients between the fasting glucose and the CIG approached unity and were only slightly lower between these tests and the 2-h OGTT. Although additional factors unrelated to the homeostatic control of plasma glucose, such as variable gastric emptying, would contribute to this, the relatively high overall intercorrelations and the simplicity and cheapness of the FPG would recommend this as the measure of choice for the assessment of glucose tolerance.

When two tests have an underlying structural relationship between their measurements (or transformations of these) that is linear, it can be instructive to determine the equation of the "line of equivalence." Linear regression, although often used, is unsatisfactory since it assumes perfect precision in the independent variable and is subject to regression dilution. There have been many approaches to deriving an unbiased estimation, as comprehensively reviewed by Riggs et al. (27), and the "weighted least squared perpendicular distance" approach (Riggs' "PW" method) has been used in this paper. In the present study, the assumption of linearity, within the limits set by the imprecision of the tests, was supported by visual inspection of plots of the relationships (data not shown). The FPG threshold concentrations recommended by the American Diabetes Association (4), based on studies of the prevalence of retinopathy in three distinct populations, were confirmed as equivalent to the established OGTT thresholds for IGT and diabetes.

We also calculated the SD of log_e(DR) estimates for different numbers of replicate tests, using the Taylor series expansion, subject to a constraint on the total number of tests performed. For a study comparing two methods using a total of 60 tests, the power to detect a difference between DRs of 2.5 and 4.0 is 58% using 30 subjects and two replicate tests, rising to 72% with 20 subjects and three tests, an increase in power of 24%. Further increasing the number of replicates gives smaller increases in power, e.g., for four tests in 15 subjects the power is 78%, with even smaller gains for more than four replicates. There would appear to be some advantage in using three, rather than two, replicate tests in each subject, but there is little advantage in increasing the number beyond three given the need to obtain sufficient subjects to adequately cover the range of interest.

This study showed that, with OGTT done under carefully controlled conditions, with a reproducibility that is somewhat better than reported elsewhere, it was not possible to distinguish between the WHO thresholds for IGT and diabetes at a 5% significance level, and this was also the case for FPG, even when the mean of three samples at 5-min intervals was assessed (the between-sample variation being relatively small in relation to the between-day variation). It is therefore not surprising, with two thresholds close together, that repeat measurements often give change of status. Improved classification could be achieved by taking the mean of determinations on more than one day. However, although these classifications may be useful for epidemiological purposes, for practical purposes the actual OGTT or FPG value is more informative (31, 32).

In summary, we have outlined a comprehensive but simple methodology for the comparison of imprecise tests, encouraging 1) comparison of test discrimination, expressed as the DR, 2) the evaluation of the degree of agreement between tests based on correlation coefficients adjusted for attenuation, and 3) in the case of a linear relationship between test results (or their mathematical transformations), the use of an unbiased method for estimating the underlying equation. For such a comparison study, it is important to determine the within-subject variation of each test as well as the variation between subjects. Application of these methods to various tests of glucose tolerance demonstrated similar discrimination, acceptable agreement, and an unbiased estimation of the FPG values equivalent to those of the 2-h OGTT. The latter agree closely with the outcome-derived thresholds currently being recommended by the American Diabetes Association. However, because the thresholds for IGT and diabetes are within measurement error and cannot be reliably distinguished, the absolute 2-h OGTT or FPG is more informative than the categorization.