Linear Global Sensitivity via Regression and Correlation [DRAFT]

Part of a series: Sensitivity analysis.

Follow reading here

Linear Surrogate Modeling

When dealing with systems known to be linear, a common approach is to approximate the full model with a linear surrogate. This surrogate model can be represented as:

\[ y = AX \]

Here, \(y\) is the output, \(X\) is the input vector, and \(A\) represents the coefficient matrix.

To further simplify the linear surrogate, a linear approximation via regression is performed on the output. The resulting model takes the form:

\[ y = \sum_i \beta_i x_i \]

Here, \(\beta_i\) denotes the coefficients obtained from the linear regression, representing the global effects.

Assessing Model Quality

The coefficient of determination, \(R^2\), serves as a crucial metric to evaluate the quality of the linear model. To enhance the interpretability of the results, data normalization is employed. The empirical mean (\(E[x]\)) and variance (\(V[x]\)) are used to normalize both the input and output data:

\[ \tilde{x_i} = \frac{x_i - E[x_i]}{V[x_i]} \]

\[ \tilde{y_i} = \frac{y_i - E[x_i]}{V[x_i]} \]

Normalizing the solutions ensures comparability between different models and eliminates the effects of units on the coefficients. The resulting coefficients, known as Standard Regression Coefficients (SRC), capture the mean effects. Additionally, it can be observed that:

\[ V(y) = \sum_i \beta_i^2 x_i \]

This expression provides a measure of variance due to \(x_i\), and the Standard Regression Coefficients can be expressed as:

\[ SRC_i = \beta_i \sqrt{\frac{V[x_i]}{V[y]}} \]

Global Sensitivity Measures

In the realm of linear regression models, two other global sensitivity measures are prominent: the Correlation Coefficient (CC) and the Partial Correlation Coefficient (PCC).

The Correlation Coefficient (\(CC_i\)) is given by:

\[ CC_i = \frac{COV(x_i,y)}{\sqrt{V[x_i]V[y]}} \]

On the other hand, the Partial Correlation Coefficient (\(PCC_{i|S_i}\)) removes the linear effects of other terms. For a variable \(x_i\), it is expressed as:

\[ PCC_{i|S_i} = \frac{COV(x_i, y | S_i)}{\sqrt{V[x_i|S_i] V[y| S_i]}} \]

Here, \(S_i = \{ x_1,x_2, \ldots ,x_{i-1}, x_{i+1}, \ldots, x_n \}\).

For a more in-depth exploration and analysis, refer to [Arriola and Hyman, 2009]. on sensitivity analysis.

Conclusion

Understanding the intricacies of sensitivity analysis in linear systems is essential for unraveling the dynamics of complex models. The techniques discussed here, including linear surrogate modeling and global sensitivity measures, provide valuable insights into the impact of individual variables on the system’s behavior.

References

AH09

Leon Arriola and James M Hyman. Sensitivity analysis for uncertainty quantification in mathematical models. Mathematical and statistical estimation approaches in epidemiology, pages 195–247, 2009.

Authors

Maqsood Mubarak Rajput, Kainat Khowaja