Post History
#3: Post edited
by
clemens
·
2026-05-03T20:49:35Z (5 months ago)
- Why, yes. If one has a linear regression $\mathbf{y} = \mathbf{X} \mathbf{\beta} + \mathbf{\epsilon}$ and models the errors as i.i.d Gaussians with known standard deviation $\sigma$ then the likelihood function for the "true" coefficients will be a multivariate Gaussian with that standard deviation.
- The covariance matrix that determines this multivariate Gaussian will be the same as the covariance matrix of the columns of $X$. This is because we are orthogonally projecting the spherical Gaussian distribution surrounding $y$ onto the hyperplane defined by $X$.
- Of course for less than 30 data points we don't know the right standard deviation and so we have to use a T-distribution for our likelihood function.
As for the OP's particular question of how to test whether one coefficient is greater than another: for large samples, all we need to do is run our linear regression, then run it again with the two coefficients "pegged" to equal each other, and look at the relative increase in the standard deviation of our error estimate; that gives us our test statistic which we can plug into the one-tailed Z-test. (Briefly, this computes *how far*, in units of standard deviations of the error, the line $\beta_1=\beta_2$ is from the estimated parameters.)
- Why, yes. If one has a linear regression $\mathbf{y} = \mathbf{X} \mathbf{\beta} + \mathbf{\epsilon}$ and models the errors as i.i.d Gaussians with known standard deviation $\sigma$ then the likelihood function for the "true" coefficients will be a multivariate Gaussian with that standard deviation.
- The covariance matrix that determines this multivariate Gaussian will be the same as the covariance matrix of the columns of $X$. This is because we are orthogonally projecting the spherical Gaussian distribution surrounding $y$ onto the hyperplane defined by $X$.
- Of course for less than 30 data points we don't know the right standard deviation and so we have to use a T-distribution for our likelihood function.
- As for the OP's particular question of how to test whether one coefficient is greater than another: for large samples, all we need to do is run our linear regression, then run it again with the two coefficients "pegged" to equal each other (by merging two columns of the $\mathbf{X}$ matrix), and look at the relative increase in the standard deviation of our error estimate; that gives us our test statistic which we can plug into the one-tailed Z-test. (Briefly, this computes *how far*, in units of standard deviations of the error, the line $\beta_1=\beta_2$ is from the estimated parameters.)
- The same approach works for testing whether one coefficient is greater or less than any linear combination of the other coefficients.
#2: Post edited
by
clemens
·
2026-05-03T03:15:16Z (5 months ago)
Added particular case of the problem
Why, yes. If one has a linear regression $\mathbb{y} = \mathbb{X} \mathbb{\beta} + \mathbb{\epsilon}$ and models the errors as i.i.d Gaussians with known standard deviation $\sigma$ then the likelihood function for the coefficients will be a multivariate Gaussian with that standard deviation.- The covariance matrix that determines this multivariate Gaussian will be the same as the covariance matrix of the columns of $X$. This is because we are orthogonally projecting the spherical Gaussian distribution surrounding $y$ onto the hyperplane defined by $X$.
Of course for less than 30 data points we don't know the right standard deviation and so we have to use a T-distribution for our likelihood function.
- Why, yes. If one has a linear regression $\mathbf{y} = \mathbf{X} \mathbf{\beta} + \mathbf{\epsilon}$ and models the errors as i.i.d Gaussians with known standard deviation $\sigma$ then the likelihood function for the "true" coefficients will be a multivariate Gaussian with that standard deviation.
- The covariance matrix that determines this multivariate Gaussian will be the same as the covariance matrix of the columns of $X$. This is because we are orthogonally projecting the spherical Gaussian distribution surrounding $y$ onto the hyperplane defined by $X$.
- Of course for less than 30 data points we don't know the right standard deviation and so we have to use a T-distribution for our likelihood function.
- As for the OP's particular question of how to test whether one coefficient is greater than another: for large samples, all we need to do is run our linear regression, then run it again with the two coefficients "pegged" to equal each other, and look at the relative increase in the standard deviation of our error estimate; that gives us our test statistic which we can plug into the one-tailed Z-test. (Briefly, this computes *how far*, in units of standard deviations of the error, the line $\beta_1=\beta_2$ is from the estimated parameters.)
#1: Initial revision
by
clemens
·
2026-05-03T03:00:54Z (5 months ago)
Why, yes. If one has a linear regression $\mathbb{y} = \mathbb{X} \mathbb{\beta} + \mathbb{\epsilon}$ and models the errors as i.i.d Gaussians with known standard deviation $\sigma$ then the likelihood function for the coefficients will be a multivariate Gaussian with that standard deviation.
The covariance matrix that determines this multivariate Gaussian will be the same as the covariance matrix of the columns of $X$. This is because we are orthogonally projecting the spherical Gaussian distribution surrounding $y$ onto the hyperplane defined by $X$.
Of course for less than 30 data points we don't know the right standard deviation and so we have to use a T-distribution for our likelihood function.