## Resources

Vandenberghe Lectures

## Notation

variable dimension name
$\mathbf{y}_i$ $\mathbb{R}^{(M \times 1)}$ ith predictor
$\mathbf{x}_i$ $\mathbb{R}^{(P \times 1)}$ ith state
$\alpha_i$ $\mathbb{R}^{(1 \times 1)}$ ith sample weight
$\mathbf{w}$ $\mathbb{R}^{(P \times M)}$ weights
$\mathbf{Y}$ $\mathbb{R}^{(N \times M)}$ all predictors
$\mathbf{X}$ $\mathbb{R}^{(N \times P)}$ all states
$\boldsymbol{\alpha}$ $\mathbb{R}^{(N \times N)}$ identity matrix with columns entries being data point weights

## Weighted Gaussian Linear regression

The log-likelihood of dataset with $N$ weighted samples $\mathcal{D} = \{\mathbf{x}_i, \mathbf{y}_i, \alpha_i \}_{i=1:N}$ which is modeled by a linear gaussian function is given by:

$$L(\mathcal{D};\boldsymbol{\theta}) \triangleq \sum_{i=1}^N \log p(\mathbf{y}_i|\mathbf{x}_i;\boldsymbol{\theta}) \, \alpha_i$$

where $p(\cdot)$ is a Gaussian probability density function:

with parameters $\boldsymbol{\theta} \triangleq \{\mathbf{w}, \boldsymbol{\Sigma}\}$.

### Expansion of the log-likelihood

First without considering the weights $\alpha$ we simplify $L(\mathcal{D};\boldsymbol{\theta})$

Simplifying with the weights:

## 1D Maximum likelihood

Given that we are in the 1D case $\boldsymbol{\Sigma} = \sigma$, $D=1$

Set the derivaties with respect to the parameters to zero, $\frac{\partial L}{\partial \mathbf{w}} = 0$ and $\frac{\partial L}{\partial \sigma^2} = 0$, and solve for $\mathbf{w}$ and $\sigma^2$: