Statistical methods for analysis of HTS RNAi screenning

RNAi screening analysis workflow

Step 1: data triage

Step 2: normalization

Fraction or percentage of control

common approach

division each sample value by the mean of the control (either positive or negative)

requires a large number of controls to provide adequate estimation of their mean

sensitive to outliers

Fraction or percent of samples

              common approach

the mean of samples on the plate may be substituted for the means of the controls

reduce the need for large numbers of controls

exacerbate the issue of nonrobustness

cannot incorporate information on the degree of variation in the sample data

z score and robust z score

              z score: the number of standard deviations from the mean

              frequently used in RNAi screening

incorporate information on the degree of variation in the sample data

              depends on the use of samples as de facto negative controls

z score is sensitive to outliers-> robust z score, substitutes the outlier-insensitive median and median absolute deviation for mean and standard deviation in the z-score calculation

B score

              for within-plate systematic effects

applied to remove row, column or well effects by iterative application of the Tukey median polish algorism

relatively robust to outliers

essentially uses the samples as negative controls

              cellHT2 package in BioConductor bioinformatics software

Step 3: calculation of quality metrics

Z’ or Z factor

              The most common quality metrics for RNAi and small-molecule screens

              Z’ factor is often used during assay optimization (based on controls)

              Z factor may be used to assess performance of the screen on actual samples

Strictly standardized mean difference (SSMD)

              developed for use with RNAi screening

              more rigorous than Z factor

the ratio between the difference of the means and the standard deviation of the difference between two populations

In one case, accurately captured the clear difference between the high and low populations, which is not detected by Z’ factor

Receiver operating characteristic curves (ROC)

              used as quality metric in microarray transcriptomics

              provides quick and intuitive understanding of dynamic ranges

multiple thresholds for defining positives and the resulting trade-offs between sensitivity and specificity can easily be investigated by plotting multiple ROC curve

used to compare validation performance of hits generated from differently normalized RNAi data

Step 4: hit identification

Median ±k median absolute deviation (MAD)

              an improvement of mean ±k standard deviation approach

              identify both strong and weak hits while controlling false positives

robust to outliers and to identify weak hits in RNAi data effectively

              generate fewer false negatives than mean ±kstandard deviation

              very easy to calculate

Multiple t-tests

simple to implement and understand, but requires three or more replicates of each condition

If a high false positive rate cannot be tolerated, it is imperative to apply multiple-comparison corrections to the resulting P-values of each individual test, resulting sensitive to outlier

Quartile-based selection

              for the unsymmetrical data distribution

set upper and lower hits selection thresholds based on number of interquartile ranges above or below the first and third quartiles of the data

              identify both strong and weak hits while controlling false positives

easy to calculate, but has not been general in RNAi screening (MAD is more common)

SSMD for hit identification

calculating the SSMD limits for hit selection based on the desired false positive rate, false negative rate or both

requires many negative controls

not calculated by standard analysis packages

Redundant siRNA activity (RSA)

              integrate information about multiple RNAi reagents tested for each gene

ranks silencing reagents according to experimental effect and assigns a P-value to all reagents for single gene based on whether the reagents for that gene are distributed significantly higher in the rankings than would be expected by chance

be able to provide P-value for gene hits without sacrificing robustness

have higher rates of reconfirmation than those of identified with conventional methods

Although RSA is not included in common analysis software packages, ist developers have made available implementations in C# (for Windows), R and Perl

Rank product

              originally developed for use with microarray data

The premise of the rank-product approach is that a consistent hit should be highly ranked in each independent biological replicate set.

provides P-values for potential hits without requiring the assumption of an underlying probability distribution

              requires substantial computation and several replicates per screen to work

similar to RSA, but it does not depend on the use of multiple different RNAi reagents per gene

available as part of RNAither package in BioConductor

Bayesian models

To seek explicit estimated probabilities that a given siRNA has no effect, and inhibition effect or an activation effect

2 models are reported in 2008

simpler model; using only the negative controls to describe the posterior distribution of the ne true mean value for sample given the observed data value

more complex model; a posterior distribution that assumes the availability of data from both positive-inhibition and positive activation controls as well as negative controls

Both model provide the means to calculate the false discovery rate associated with any given hit threshold, but are usable only on screens without replicates

incorporates both plate-wide and experiment-wide information as well as information from both negative controls and the assumed de facto negative samples

simpler Bayesian model is followed by plate-wide MAD

not yet available in common software