A short comment on Stat paper Griswold et al. (2013): Practical marginalized multilevel models, by Jan Gertheiss, University of Göttingen

Inspired by the discussion on StatBlog on open peer review and the idea of viewing every paper as a discussion paper, I would like to take the opportunity to make a short comment on a paper recently published in Stat: Practical marginalized multilevel models by Griswold et al. (2013). This is a very interesting paper reformulating the marginalized multilevel model (MMM; Heagerty and Zeger, 2000) and showing how to fi t such a model using existing mixed model computing solutions. The MMM nicely combines marginal and conditional models by specifying both a marginal mean model and a conditional model representing the association structure of longitudinal or other clustered data.

When I was reading Section 4, which describes two real data examples, I looked at the fi tted marginal models and wondered how results would look like if the simplest fitting approach had been used: a marginal model with working independence. For example, I analyzed the visual impairment data and used the R gee package (Carey, 2012) to fit a marginal logistic model with working independence. Results are given below. Thanks to Bruce Swihart for providing the data and related information. The data is now also available at http://www.biostat.jhsph.edu/~bswihart/Publications/.

Coefficients:                 
                Estimate   Naive S.E.      Naive z   Robust S.E.    Robust z  
(Intercept)   -2.4398758   0.04872570   -50.073693    0.05768490   -42.29661  
black          0.1005937   0.07096382     1.417535    0.08521642     1.18045

We see that the fitted coefficients diff er slightly from those given in the paper’s Table IV. Taking the (robust) standard errors into account, however, these differences cannot be seen as substantial, confirming the results from the paper. The standard errors (both in the paper and above) also indicate that there is no significant marginal effect of race. However, this only due to the fact that other covariates are ignored. In particular when age and education is taken into account, there is a clear effect of race; see, for example, Liang and Zeger (1993).

References

  1. Carey, V. J. (2012). gee: Generalized Estimation Equation solver. R package version 4.13-18; ported to R by Thomas Lumley and Brian Ripley.
  2. Griswold, M. E., B. J. Swihart, B. S. Ca o, and S. L. Zeger (2013). Practical marginalized multilevel models. Stat 2, 129-142.
  3. Heagerty, P. J. and S. L. Zeger (2000). Marginalized multilevel models and likelihood inference. Statistical Science 15, 1-19.
  4. Liang, K. Y. and S. L. Zeger (1993). Regression analysis for correlated data. Annual Reviews of Public Health 14, 43-68.

Leave a comment

Filed under Uncategorized

Open peer discussion: An alternative to closed peer review by Peter Hoff, University of Washington

 Journals and their alternatives

Our current journal publication system has traditionally served two purposes: dissemination and peer review. Regarding dissemination, the journal system arguably does a good job with some things (special issues, discussion pieces, copy-editing), but does a poor job with others (open-access, time to publication, article updating). Happily, we have alternative ways of disseminating our research. For example, the arXiv provides an updatable, open-access article platform that appears publicly within days of article submission. Furthermore, posting an article on the arXiv does not preclude it from being published in a journal. In this way, the arXiv can be seen as a complement to the journal system, rather than just an alternative or replacement.

The standard argument against replacing the journal system with something like the arXiv has been that the journal system provides peer review. However, the journal peer review system is deficient in many ways and, unlike the case of dissemination, there is not currently a viable alternative (or complementary) system. It is worthwhile, then, to consider what an alternative system would look like, and whether or not it would be feasible to implement.

Problems with peer review

In addition to being an author and reviewer, I am an associate editor for three statistics journals. My effectiveness as an AE is limited by many aspects of the peer review system, three of which include 1) the unrepresentative nature of the reviews, 2) that the reviews, and typically any insights or criticisms made by the reviewers, are never made public, and 3) that the link between the review and the publication decision lowers the level of discussion in the review process.

Regarding the first problem, my most challenging task as an AE is finding capable reviewers who are both interested and willing to do a review within a particular time-frame. Even in the ideal scenario where two interested and capable reviewers can be obtained, it is unlikely that such a small number of reviews will reflect the diversity of opinion about the article in the broader community.

Regarding the confidential nature of reviews: Good reviews often provide insights, critiques and cautions that are only addressed in the publication superficially, if at all. It is wasteful that these potentially valuable insights are not made available to the potential readers of the article.

Finally, I would theorize that the negativity and unhelpfulness of many reviews stems from the fact that such reviews are essentially arguments for the reviewers recommendation: A reviewer decides in their mind if a paper is good enough for the journal, and if not, writes a review trying to prove this position. Since reviewers rarely recommend “accept as is”, the review often becomes a critique trying to prove the seriousness of the reviewer-identified flaws, rather than a commentary on the main ideas or results of the paper.

An Open Peer Discussion System

Could these problems be solved? Could we have a system where people review articles of their own choosing, on a time-frame that works for them? Could reviews be made public, so that the reader of an article is aware of comments or critiques of others? Could we eliminate the link between reviews and publication, in the hope that reviews would more closely resemble a discussion of the article, rather than an argument that the article needs to be rewritten in a way that is more related to the reviewer’s interests?

Consider a system in which, in order to get feedback on your article (i.e., to have your article “discussed”), you would need to discuss two other articles. One of these you would choose with complete freedom, and the other you would choose from a list of unreviewed papers, or perhaps would be assigned (based on bibliographic comparisons or stated areas of expertise). Such a requirement would not be much of a burden – presumably the article you choose would be one that you’ve already read, or want to read. This “reviewer choice” aspect of the system would mean that interesting, important or controversial articles would be heavily and publicly discussed, while the “reviewer assigned” aspect would maintain a minimal level of review for all articles in the system.

Perhaps reviews would be anonymous, perhaps not. However, note that people happily sign their names to critical comments on discussion papers found in journals. A thoughtful but critical review on an open peer discussion system could raise the visibility and prestige of the reviewer/discussant. Conversely, having an article discussed by many would perhaps bring more visibility than simply having the article appear undiscussed in a journal. In a sense, every paper in an open peer discussion system is a potential discussion paper. Discussions may be viewed by all, and contributed to by all participants in the system.

Summary

Developing technologies are quickly changing the ways in which we disseminate, encounter and access scientific research. In this changing environment, it is healthy to have multiple ways of disseminating and evaluating our work. Having alternatives will help us avoid being trapped by a outdated system that has long held a monopoly over our primary means of scholarly discussion. Good alternatives exist for the purpose of disseminating research. What remains for us to do is to develop an alternative means of discussing and evaluating our work.

3 Comments

Filed under Uncategorized

Modern Publication by David Banks, Duke University

All my points have been made before, by smarter people.  But it is worthwhile to sort through them again—the world needs to change, and steady persuasion is slowly working.

I, and many others, believe that the traditional model for academic publishing is broken.  But modern technology offers fresh solutions.  My views on this are my own, but they are informed by the opinions of many others; Nick Fisher, Andrew Gelman, Peter Hoff, Nick Jewell, Richard Levine, Jim Pitman, Karl Rohe, Len Stefanski, Andrew Thomas, and Larry Wasserman are prominent voices.

The first, and easiest, issue is distribution.  For centuries, the best way to exchange ideas was to paint them on thinly sliced trees and mail them to each other.  I think we can all agree that this is quaint; each year, more researchers and libraries shift to electronic access.  The storage costs for paper are unsustainable, and hardcopy access is inconvenient. I can find an article faster on-line than I can find it on my bookshelf.

The second issue is delay. The time from submission to appearance in most of our hardcopy journals is about two years.  For electronic journals, it takes about one year. This is slower dissemination than happens in rapidly moving fields such as biology and computer science.  To address that problem, a number of electronic workarounds have arisen: arXiv posts unrefereed material immediately; authors now routinely put drafts on their website; STAT makes enormous effort to achieve a two-week review time, with an up-or-down decision on publication.  All of these are valuable models, and which of these, or which cocktail among them, will persist is unclear.

The most controversial issue concerns the costs and benefits of refereeing. For nearly all cases, peer review improves a paper.  Chopin, Gelman, Robert and Mengersen (2012) eloquently champion traditional review.  But in self-destructive irony, I note that their paper appeared in the unrefereed outlet arXiv, and had I been invited to review their submission, I would have asked for a fuller account of the costs the profession pays for achieving such improvement.

Based on my own experience as an editor, referee and author, I believe the tradeoffs are as follows.  Sometimes, review leads to critical improvement. But in many cases, the improvement is minor.  Against this, there is the delay imposed by review, and the burden that refereeing places upon our researchers.  Every refereed paper requires hours of volunteer time from two domain experts, an associate editor, and an editor.  If a paper is rejected and then submitted to another journal, the burden doubles. Recently, there has been a proliferation of journals, and new researchers are expected to publish more papers than previous academic generations.  The upshot of these trends is that the quality of refereeing has eroded and the pool of volunteer talent has been depleted.

Additionally, the review process is noisy.  When I was an editor of the Journal of the American Statistical Association, I know of one paper that I incorrectly rejected, and there are at least two other cases that were probably Type I errors as well.  And I prefer not to think about the Type II mistakes.  Such errors are almost inescapable—the first-choice referee is too busy and prominent to have time to review.  The second-choice referee is really good, but has already reviewed three papers this semester and one
cannot impose on her again.  The third-best referee is at the author’s institution.  The fourth best referee just had a baby.  The fifth-best referee declines for reasons that are obscure.  So one finally recruits the sixth-best and seventh-best referees, and the reports are superficial, mostly pointing out typos and and quibbling over the scope of the simulation study.

Modern publication offers various ameliorative solutions.  First, one can publish living documents.  When mistakes are found in a paper, or better wording is wanted, the changes can be made and the paper reposted. This seems far superior to traditional publication, in which the version-of-record is fixed by corrigenda (even in electronic journals).  Second, there is the opportunity for on-line discussion of papers and visible referee reports—if a referee knows their work will be seen by the community, and perhaps even attributed, then I anticipate better quality (but fewer volunteers).  Third, Internet technology has developed recommender systems and collaborative filtering tools that provide indications of the quality of a publication.  So the oft-voiced concern that without refereeing, readers would be unable to separate wheat from chaff, is misplaced.  The Web is full of trash, but rarely are we misled into conspiracy theories, Nigerian inheritances, or ineffective sexual enhancements.

There is much more, of course. Electronic publication will prompt more access to code and possibly data, and facilitate reproducibilty studies and better research standards.  It could rebalance the economics of publishing, so that academics don’t work for free to generate revenue for publishers, There is the opportunity to make the publication process easier for non-native writers of English.  And it could create a culture with more accountability and transparency, while reducing the intellectual stoop labor of traditional refereeing.

I don’t want to advocate a particular solution—I have ideas, but others may have better ones.  Instead, I urge the profession to rethink this whole process from the ground up.

Leave a comment

Filed under Uncategorized

Shrinkage Priors: Computational vs. Statistical Efficiency by Natesh Pillai, Harvard University

We discuss an example of statistical efficiency vs. computational efficiency in the context of shrinkage priors in some high dimensional problems. Most of the material is from the author’s recent joint work (Bhattacharay et al., 2012). The author takes this opportunity to thank his collaborators Anirban Bhattacharya, Debdeep Pati and David Dunson for sharing their valuable insights. He also thanks Andrew Gelman, Eric Laber, Abel Rodriguez and James Scott for helpful discussions, but is solely responsibility for the contents of this blog.

Introduction
Penalization methods in high dimensional problems, being the flavor of the last decade, have enjoyed much attention resulting in a rich
theoretical literature establishing their optimality properties. There
are also fast algorithms and compelling applied results underlining the
success story of these methods. The overwhelming emphasis has been on
rapidly producing a point estimator with good empirical and theoretical
properties.  However, to the best of this author’s knowledge, a
satisfactory theory for uncertainty quantification for penalized
estimators is not yet in place.

 Given that most shrinkage estimators correspond to the mode of a
Bayesian posterior, it is natural to ask whether we can use the whole
posterior distribution to provide a probabilistic measure of
uncertainty. The hope is to be able to choose a default shrinkage
prior that leads to similar optimality properties to those shown for
L_1 penalization and other approaches, but with the added advantage of the entire posterior distribution concentrating at the
optimal rate, instead of just focusing on the point estimate. Furthermore, taking a Bayesian perspective has distinct advantages in terms of tuning parameter choice, allowing key penalty parameters to be marginalized over the posterior distribution instead of relying on cross-validation. Of course, for the Bayesian methods to be useful for uncertainty quantification, there should exist fast Markov Chain Monte Carlo (MCMC) algorithms to explore these gigantic model spaces. As life would have it, there are theoretically efficient priors for the MCMC algorithm which are slow in high dimensions, and conversely, there are priors for which there exist fast computational algorithms but are not theoretically efficient.  The
point is further detailed in a concrete example below.

 Point mass priors vs. Shrinkage Priors
For a high-dimensional vector \theta \in \mathbb{R}^n, a natural, time-tested way to incorporate sparsity in a Bayesian framework is to use point mass mixture priors\displaystyle \theta_j \sim (1 - \pi) \delta_0 + \pi g_{\theta}, \quad j = 1, \ldots, n,     (1)

where \pi = \mbox{Pr}(\theta_j \neq 0), \mathbb{E}\{  |\mbox{supp}(\theta)| \mid \pi\} = n \pi is the prior guess on model
size (sparsity level), and g_{\theta} is an absolutely continuous
density on \mathbb{R}.  It is common to place a beta prior on \pi,
leading to a beta-Bernoulli prior on the model size, which conveys an
automatic multiplicity adjustment (Scott and Berger, 2010).  Let us also
take the model to be, the popular Normal means model (though the
discussion here holds in more generality):

\displaystyle y_i = \theta_i + \epsilon_i, \quad \epsilon_i \sim \mbox{N}(0, 1), \quad 1 \leq i \leq n.     (2)

For the model above, Castillo and van der Vaart (2012) established that the posterior corresponding to the prior (1) with an
appropriate beta prior on \pi contracts at the frequentist minimax
rate. Thus at least theoretically, the prior given in
(1) is “hard to beat”. Unfortunately, to simulate
posterior draws corresponding to such priors is not an easy task as the MCMC algorithm has to explore a model space of dimension O(2^p). This is not feasible even for moderate p.

Shrinkage priors (there is an amazing variety of them), often constructed as continuous scale mixtures of Gaussians, are appealing alternatives as they can potentially lead to dramatically more efficient posterior computation. They also allow separate control of the level of sparsity and the size of the signal coefficients. Thus, one formulation of the million dollar question is: What family of shrinkage priors achieves the theoretical efficiency of the point mass priors?

In the recent preprint (Bhattacharya et al., 2012), we give a reasonably satisfactory answer for the above question. One of the main results is that a broad class of Gaussian scale mixture priors, including the
Bayesian Lasso and other commonly used choices such as ridge regression, are sub-optimal for the normal means model, i.e., they do not achieve the same theoretical efficiency as that of the point mass priors.

The key insight obtained in Bhattacharya et al. (2012) is that, the sub-optimal performance of shrinkage priors is due to the lack of borrowing information across coordinates apriori. Indeed, as Polson and Scott (2010) noted, essentially all such shrinkage priors can be represented as global-local (GL) mixtures of Gaussians,

\displaystyle \theta_j \sim \mbox{N}(0, \psi_j \tau), \quad \psi_j \sim f, \quad  \tau \sim g, (3)

where \tau controls global shrinkage towards the origin while the local scales \{ \psi_j \} allow deviations in the degree of shrinkage. Thus the local scales \psi_j are i.i.d from f; this lack of borrowing from other coordinates hurts the performance.

This is a well documented phenomenon in a related context. Indeed, the famous James-Stein estimator is an example of a superior estimator constructed by borrowing information across coordinates. Recall that this estimator can be considered as the posterior mean of a Bayesian model. Similarly, a high dimensional shrinkage prior which does not impose sufficient dependence across coordinates is not expected to perform well. There is ample evidence of this in Bhattacharya et al. (2012). Note that because of the global scale $\tau$, marginally the coordinate \theta_j are not independent, but it turns out that the dependence induced by \tau is often weak.

Inspired by the idea of inducing dependence, a new class of priors, called Dirichlet-Laplace (DL) priors, were proposed in Bhattacharya et al. (2012). These priors seem to achieve both optimal theoretical efficiency and efficient posterior simulation. Initial results for DL priors relative to a variety of competitors show promise, though much remains to be done. The DL prior can be written as

\displaystyle  \theta_j \sim \mathrm{DE}(\phi_j \tau), \quad \phi \sim \mathrm{Dir}(a, \ldots, a), \quad \tau \sim g     (4)

where g is a suitable density (say the half-Cauchy; see Gelman (2006)). Clearly, the DL prior induces a much stronger dependence at the local scale via the Dirichlet distribution. There
is also an efficient Gibbs sampler which leads to fast posterior simulation. Let me also add that, among the existing shrinkage priors, the horseshoe prior Carvalho et al. (2010) stands out and is very similar in performance to our DL prior. But the horseshoe seems much harder to analyze theoretically.

Have we answered the $1,000,000 Question?
Not yet! What we have done amounts to constructing a shrinkage prior which seems to achieve the same theoretical efficiency as of the point-mass priors. However, the second piece of the puzzle-computational efficiency – is not rigorously studied in the Bayesian shrinkage prior circles. The key reason for this, in my opinion, is the lack of availability of easy to use criteria to measure computational efficiency. For instance, statistical efficiency can be measured in terms of minimax rate, posterior convergence rate, various loss functions etc. On the other hand, for measuring computational efficiency of MCMC algorithms, the only prevalent notion is that of effective sample size (ESS). For most MCMC algorithms, to get exact ESS theoretically is almost impossible for real problems; but even obtaining reasonable bounds is out of reach using state of the art methods.

My point here is that to rigorously answer the efficiency trade-off, some quantitative notion of computational efficiency should be part of the equation. For instance, with in a fixed computational budget, how do the effective sample sizes corresponding two different shrinkage priors with the same statistical efficiency compare? Fortunately, this can be done by simulation and is a low hanging fruit. Again, this question is not often seriously pursued because the notion of fixed computational budget is artificial for moderate sized problems. But for really gigantic problems, which seem to be the flavor of the next decade, I believe the notion of analyzing algorithms with a fixed computational budget and incorporating them into the balance sheet of efficiency for Bayesian procedures will be an important one.

[1] Anirban Bhattacharya, Debdeep Pati, Natesh S Pillai, and David B Dunson. Bayesian shrinkage. arXiv preprint arXiv:1212.6088, 2012.
[2] C.M. Carvalho, N.G. Polson, and J.G. Scott. The horseshoe estimator for sparse signals. Biometrika, 97(2):465–480, 2010. 4
[3] I. Castillo and A. van der Vaart. Needles and straws in a haystack: Posterior concentration for possibly sparse sequences. The Annals of Statistics,
40(4):2069–2101, 2012.
[4] A. Gelman. Prior distributions for variance parameters in hierarchical models (comment on article by browne and draper). Bayesian analysis, 1(3):515–534, 2006.
[5] N.G. Polson and J.G. Scott. Shrink globally, act locally: Sparse Bayesian regularization and prediction. In Bayesian Statistics 9 (J.M. Bernardo, M.J.
Bayarri, J.O. Berger, A.P. Dawid, D. Heckerman, A.F.M. Smith and M. West, eds.), pages 501–538. Oxford University Press, New York, 2010.
[6] J.G. Scott and J.O. Berger. Bayes and empirical-bayes multiplicity adjustment in the variable-selection problem. The Annals of Statistics, 38(5):2587–2619,
2010.

Leave a comment

Filed under Uncategorized

Referee comments for “A direct sampler for G-Wishart variates” by Alex Lenkoski

The original article can be found here: http://onlinelibrary.wiley.com/doi/10.1002/sta4.23/abstract

Comment 1. Yunxiao Chen, Columbia University in the City of New York, Department of Statistics

First of all, the ideas of the double reversible jump algorithm and the analog proposed by Wang & Li (2012) are the same. Since the goal of the algorithm is to determine the model probability, it’s unclear whether your double reversible jump algorithm performs better than the algorithm proposed by Wang & Li. In my opinion, the direct sampler requires iterations to sample one point, which could make the algorithm quite slow. I guess one advantage of your direct sampler over Wang & Li’s method could be due to the fact that in a high dimensional case, when finding the clique decomposition is computationally expensive, the direct sampler can avoid it by solving the convex optimization problem while the block Gibbs sampler relies on the clique decomposition.
Second, both examples are given to show the validity of the proposed methods, and they do not show the advantage of the proposed method over other samplers. Examples/simulation studies are suggested to be included to show the advantage of the direct sampler over other methods, at least in some aspects.
In sum, the computation complexity of the direct sampler could be a big problem in practice. A comparison between the direct sampler and other samplers is suggested to be included in the paper. In addition, it’s also necessary to discuss the performance of the sampler in the high dimensional case, which I think is of particular interest in Gaussian graphical models.

Comment 2. Ran He, Columbia University in the City of New York, Department of Statistics

This paper introduces a direct sampler for G-Wishart variates such that
G-Wishart distribution can be embedded in a complicated hierarchical framework.
It samples from the Wishart model and is updated via (4) to
obtain a matrix from G-Wishart distribution. Based on this direct sampler,
this paper provides an example of application of G-Wishart distribution, by
proposing a new method of moving through the space of Gaussian graphical
models in order to form a model averaged estimate of precision matrix
K. This new method, called double reversible jump, employs the exchange
algorithm to avoid the normalizing constants in (6).
The proposed direct G-Wishart sampling method provides iid samples
from the desired G-Wishart distribution rather than running a Markov chain
to obtain samples as used in the block Gibbs sampler method. A simulation
study shows the validity of the direct sampler. However, this “direct sampler”
requires updates of K(s) until they converge, and it is not clear how fast the
convergence is. Moreover, the author doesn’t provide a comparison of the
efficiency of these two methods. Furthermore, the paper provides a method
to improve the performance of the direct sampler based on the algorithm in
Hastie et al. (2009), but it is not used in the simulation study. More results
should be provided to show how well it improves the performance.
Also, the double reversible jump algorithm is supposed to be superior to the
running MC approximation of Atay-Kayis & Massam (2005) in the criterion
of computation time. However, the paper doesn’t provide the comparison of
running times of these two methods based on the same setting. It’s better
to show the comparison in order to prove the efficiency of the new method.
In summary, the direct sampler for G-Wishart variates proposed in this
paper is innovative since it considerably expands the usefulness of the G-Wishart
distribution. Therefore, it sheds light on a host of new applications
of the G-Wishart distribution in hierarchical Bayesian modeling. However,
more theoretical and simulation results should be provided to show how well
this method outperforms the existing methods.

Leave a comment

Filed under Uncategorized

Demotivated by Gene Expression Arrays

My name is Mike Wu. I am a biostatistician working in the field of statistical genomics and I’m writing this posting because I’m tired of reading yet another statistical paper that is “motivated by” or “illustrated on” the Golub gene expression data set (or any other over analyzed gene expression data set).  First of all, the Golub data set is considered abnormally clean with such strong signal that the most naïve of methods (e.g. eyeballing the numbers) does a great job.  Second, there is a lot more out there now than just gene expression data and it’s somewhat embarrassing for statisticians to continue viewing gene expression as the sum total of modern genomics.  Accordingly, this posting is aimed at describing just a couple of newer omics areas that may be of interest to methodologically oriented statisticians.

Starting with the initial development of microarrays for gene expression profiling, advances in high throughput biotechnology have revolutionized biomedical research and led to the development of a wide range of “omic” technologies such as genomics and proteomics.  The challenges posed by the dimensionality of these omic data sets combined with the limited availability of samples has in turn spurred development of new statistical methods that are useful for analyzing the data from these emerging studies.  However, although gene expression microarray experiments arguably started the revolution, the field has moved on and investigators are increasingly more interested in other technologies, yet development of statistical tools for analyzing data from such studies has lagged significantly behind the technological developments and scientific interest.  Consequently, these new and emerging areas offer tremendous opportunities for statisticians interested in high dimensional data.

An emerging area of considerable interest to biomedical investigators is metagenomics and analysis of the microbiome.  Briefly, it’s pretty well known that there are more bacteria cells (1014) than our own human cells (1013) inhabiting our bodies and only a fraction of the DNA inside our bodies is our own.  Human clinical metagenomic studies are aimed at understanding how the diversity of a microbial population (such as the bacteria in an individual’s fecal samples) and how the abundances of the microbial species are related to outcomes.  The development of new DNA sequencing technologies has enabled identification and quantification of the bacteria taxa (for the purposes of this posting, I mean the types/species of bacteria) that make up the communities.  This is a burgeoning area of interest with recent discoveries that microbial populations can play a major role in human health and the establishment of the human microbiome project.  While initial microbiome studies were small and descriptive, increasingly there is interest in association analysis and correlation with clinical or environmental variables of interest and prediction.  Nature Biotechnology (one of the Nature journals) recently published several editorials in the April 2013 issue on challenges of translation.  Given the scientific interest and the relative dearth of statistical methodology, this represents a great opportunity for statisticians willing to get their hands dirty (perhaps not the best turn of phrase when discussing bacteria and fecal samples).  Some simple statistical issues include detection limits for the taxa (since low abundance bacteria are unlikely to be observed), incorporation of phylogenetic information, testing groups of taxa (since bacterial species can be hierarchically grouped a priori into different categories), and assessment of overall diversity, not to mention difficulties arising from complex clinical or environmental variables and from different study designs.

Another area that is growing in popularity is the field of epigenetics which is concerned with changes in DNA that do not affect the actual sequence. One particular form of epigenetic variability is DNA methylation.  Modern array based technology can now simultaneously measure DNA methylation at hundreds of thousands of locations (called CpG sites) across the genome.  The data are similar to gene expression (in fact methylation often affects gene expression) in that there can be substantial signal, the measurements are continuous (between 0 and 1, but usually close to normal), methylation can be either an outcome or a predictor, and methylation differs by tissue.  The data are similar to GWAS data in the dimensionality (modern arrays consider over 480,000 CpGs along the genome), there is strong spatial correlation along the genome, and DNA methylation is somewhat heritable.  Given that there are researchers approaching the data from both the expression world and from the GWAS world, there is considerably room for cross pollination and for translation of statistical methods across paradigms.  In addition, while I’m loathe to recommend data sets purely for the sake of having a data set in a paper, for statisticians interested in testing their methodology, the array based methylation data are also relatively clean does not necessarily require extensive genomics expertise.

Beyond microbiome analysis and methylation, other areas of potential interest to statisticians include analysis of new proteomic and metabolomics studies and data from large scale toxico and pharmico genomic screens as well as a host of other emerging omics such as exposomics, lipidomics, kinomics, and so on.  Often, the startup to get into some of these data is not large yet this is an easy way to keep our methodology current and applicable to data and problems that substantive investigators care about. In short, this is a natural means of keeping our discipline relevant and ensuring its continued health and survival.

The NCBI gene expression omnibus (GEO) offers many publicly available, methylation data sets at http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GPL13534.

Leave a comment

Filed under Editorial

Some Challenges in Developing Statistical Methods for Personalized Medicine — by Wenbin Lu, Associate Professor in Statistics at NCSU

Driven by the need of developing evidence-based personalized treatment strategies in modern medicine, especially for complex diseases such as cancer, AIDS and mental diseases, the development of sophisticated statistical learning methods for personalized medicine has become a hot topic in research and has attracted a lot attention.

In a recent issue of STAT, Drs. Zhang, Tsiatis, Davidian, Zhang and Laber (2012, page 103-114) wrote an interesting paper, titled “Estimating optimal treatment regimes from a classification perspective”. This paper proposed a unified classification framework for estimating optimal treatment regimes and demonstrated that many existing methods, such as the classical regression method, the G-estimation, the robust learning method of Zhang, Tsiatis, Laber and Davidian (2012, Biometrics) and the outcome weighted learning method of Zhao, Zeng, Rush and Kosorok (2012, JASA) can be incorporated in this general framework. This new framework makes a nice contribution to the statistical learning of optimal treatment regimes.

However, there are still many challenges in developing and implementing statistical methods for personalized treatment regimes. The proposed unified classification framework in Zhang et al. (2012) is flexible, but it requires an effective way for solving the associated optimization problem, especially when the dimension of covariates is high. The discrete and non-convex natures of the proposed weighted classification losses make it difficult. It is interesting to develop an efficient mixture of integer and linear programming algorithm for the optimization. In addition, when the dimension of covariates is high, selection of predictive variables for constructing optimal treatment strategies becomes necessary and crucial to achieve personalized medicine, since it can lead to more interpretable, reliable and implementable treatment rules for practical use. Recently, Lu, Zhang and Zeng (2012, Statistical Methods in Medical Research) proposed a penalization method based on an A-learning regression framework for selecting predictive variables in optimal treatment decision. It is also interesting to study the selection of predictive variables based on the classification framework of Zhang et al. (2012). Another challenge may come from the increased number of possible treatments due to the lack of sufficient subjects consistent with a treatment strategy, for example, developing optimal time and dose combination of treatments. The current classification framework for estimation of optimal treatment regimes can only handle a fixed small number of treatments. The extensions to allow a large number of possible treatments or even infinite many treatments in a continuous fashion, require the development of new novel statistical learning methods.

Leave a comment

Filed under Editorial