CS231A

This is my blog.

CS231A笔记，有空就看看，然后更新这里啦！

（未完待续）

Note

Camera Models

Pinhole cameras

The aperture is referred to as the pinhole O or center of the camera.

The distance between the image plane and the pinhole O is the focal length f.

Sometimes, the retinal plane is placed between O and the 3D object at a distance f from O. In this case, it is called the virtual image or virtual retinal plane.

$P = [x\ y\ z]^T$ be a point on some 3D object visible to the pinhole camera. P will be mapped or projected onto the image plane Π’ , resulting in point $P’ = [x’\ y’]^T$ . Similarly, the pinhole itself can be projected onto the image plane, giving a new point C”.

The line defined by C” and O is called the optical axis of the camera system.

As the aperture size decreases, the image gets sharper, but darker.

Cameras and lenses

In modern cameras, the above conflict between crispness and brightness is mitigated by using lenses, devices that can focus or disperse light.

The corresponding projection into the image will be blurred or out of focus.

Because the paraxial refraction model(近轴折射模型) approximates using the thin lens assumption, a number of aberrations can occur. The most common one is referred to as radial distortion, which causes the image magnification to decrease or increase as a function of the distance to the optical axis. We classify the radial distortion as pincushion distortion when the magnification increases and barrel distortion when the magnification decreases. Radial distortion is caused by the fact that different portions of the lens have differing focal lengths.

Going to digital image space

k, l whose units would be something like $\frac{pixels}{cm}$, correspond to the change of units in the two axes of the image plane. Note that k and l may be different because the aspect ratio of the unit element is not guaranteed to be one.

If k = l, we often say that the camera has square pixels.

$$P'=\left[\begin{matrix}x'\\y'\end{matrix}\right]=\left[\begin{matrix}fk\frac x z+c_x\\fl\frac y z+c_y\end{matrix}\right]=\left[\begin{matrix}\alpha\frac x z+c_x\\\beta\frac y z+c_y\end{matrix}\right]$$

we see that this projection $P → P’$ is not linear,

Note that the equality between a vector and its homogeneous coordinates only occurs when the final coordinate equals one.

$$P'_h=\left[\begin{matrix}\alpha&0&c_x&0\\0&\beta&c_y&0\\0&0&1&0\end{matrix}\right]P_h=MP=K\left[\begin{matrix}I&0\end{matrix}\right]P$$

The matrix K is often referred to as the camera matrix. Two parameters are currently missing from our formulation: skewness and distortion. Most cameras have zero-skew, but some degree of skewness may occur because of sensor manufacturing errors.

Deriving the new camera matrix accounting for skewness is outside the scope of this class and we give it to you below:

$$K=\left[\begin{matrix}x'\\y'\\z\end{matrix}\right]=\left[\begin{matrix}\alpha&-\alpha cot\theta&c_x\\0&\frac \beta{sin\theta}&c_y\\0&0&1\end{matrix}\right]$$

All parameters contained in the camera matrix K are the intrinsic parameters, which change as the type of camera changes. The extrinsic paramters include the rotation and translation, which do not depend on the camera’s build.

Camera Calibration

This problem of estimating the extrinsic and intrinsic camera parameters is known as camera calibration.

We set up a linear system of equations from n correspondences such that for each correspondence $P_i , p_i$ and camera matrix M whose rows are $m_1,m_2,m_3$:

$$p_i=\left[\begin{matrix}u_i\\v_i\end{matrix}\right]=MP_i=\left[\begin{matrix}\frac{m_1P_i}{m_3P_i}\\\frac{m_2P_i}{m_3P_i}\end{matrix}\right]$$ $$u_i(m_3P_i)-m_1P_i=0\\v_i(m_3P_i)-m_2P_i=0$$

Given n of these corresponding points, the entire linear system of equations becomes

$$\left[\begin{matrix}P_1^T&0^T&-u_1P_1^T\\0^T&P_1^T&-v_1P_1^T\\\cdot&\cdot&\cdot\\\cdot&\cdot&\cdot\\\cdot&\cdot&\cdot\\P_n^T&0^T&-u_nP_n^T\\0^T&P_n^T&-v_nP_n^T\end{matrix}\right]\left[\begin{matrix}m_1^T\\m_2^T\\m_3^T\end{matrix}\right]=Pm=0$$

When 2n > 11, our homogeneous linear system is overdetermined.

We know that the camera matrix has 11 unknown parameters. This means that we need at least 6 correspondences to solve this.

If we let $P = UDV^T$ , then the solution to the above minimization is to set m equal to the last column of V . We know our SVD-solved M is known up to scale, which means that the true values of the camera matrix are some scalar multiple of M:

$$\rho M=\left[\begin{matrix}\alpha r_1^T-\alpha cot\theta r_2^T+c_xr_3^T&\alpha t_x-\alpha cot\theta t_y+c_xt_z\\\frac \beta {sin \theta}r_2^T+c_yr_3^T&\frac \beta {sin \theta}t_y+c_yt_z\\t_3^T&t_z\end{matrix}\right]$$

Here, $r_1^T,r_2^T$ and $r_3^T$ are the three rows of R.

$$M=\frac 1 \rho \left[\begin{matrix}\alpha r_1^T-\alpha cot\theta r_2^T+c_xr_3^T&\alpha t_x-\alpha cot\theta t_y+c_xt_z\\\frac \beta {sin \theta}r_2^T+c_yr_3^T&\frac \beta {sin \theta}t_y+c_yt_z\\t_3^T&t_z\end{matrix}\right]=\left[\begin{matrix}A&b\end{matrix}\right]=\left[\begin{matrix}a_1^T\\a_2^t\\a_3^T\end{matrix}\right]\left[\begin{matrix}b_1\\b_2\\b_3\end{matrix}\right]$$

Solving for the intrinsics gives

$$\rho=\pm\frac 1 {\|a_3\|}\\c_x=\rho^2(a_1\cdot a_3)\\c_y=\rho^2(a_2\cdot a_3)\\\theta=cos^{-1}(-\frac{(a_1\times a_3)\cdot(a_2\times a_3)}{\|a_1\times a_3\|\cdot\|a_2\times a_3\|})\\\alpha=\rho^2\|a_1\times a_3\|sin\theta\\\beta=\rho^2\|a_2\times a_3\|sin\theta$$

The extrinsics are

$$r_1=\frac{a_2\times a_3}{\|a_2\times a_3\|}\\r_2=r_3\times r_1\\r_3=\rho a_3\\T=\rho K^{-1}b$$

Handling Distortion in Camera Calibration

Often, distortions are radially symmetric because of the physical symmetry of the lens. We model the radial distortion with an isotropic transforma-tion:

$$QP_i=\left[\begin{matrix}\frac 1 \lambda&0&0\\0&\frac 1 \lambda&0\\0&0&\frac 1 \lambda\end{matrix}\right]MP_i=\left[\begin{matrix}u_i\\v_i\end{matrix}\right]=p_i$$

We get

$$u_iq_3P_i=q_1P_i\\v_iq_3P_i=q_2P_i$$

And before we know

$$\frac {u_i}{v_i}\frac{\frac{m_1P_i}{m_3P_i}}{\frac{m_2P_i}{m_3P_i}}=\frac{m_1P_i}{m_2P_i}$$

Similar to before, this gives a matrix-vector product that we can solve via SVD:

$$Ln=\left[\begin{matrix}v_1P_1^T&-u_1P_1^T\\\cdot&\cdot\\\cdot&\cdot\\\cdot&\cdot\\v_nP_n^T&-u_nP_n^T\end{matrix}\right]\left[\begin{matrix}m_1^T\\m_2^T\end{matrix}\right]$$

Rigid Transformations

One intuitive way to think of rotations is how much we rotate around each degree of freedom, which is often referred to as Euler angles. However, this methodology can result in what is known as singularities, or gimbal lock, in which certain configurations result in a loss of a degree of freedom for the rotation.

One way to prevent this is to use rotation matrices, which are a more general form of representing rotations. Rotation matrices are square, orthogonal matrices with determinant one.

we can represent a rotation α, β, γ around each of the respective axes as follows:

$$R_x(\alpha)=\left[\begin{matrix}1&0&0\\0&cos\alpha&-sin\alpha\\0&sin\alpha&cos\alpha\end{matrix}\right]\\ R_y(\beta)=\left[\begin{matrix}cos\beta&0&sin\beta\\0&1&0\\-sin\beta&0&cos\beta\end{matrix}\right]\\ R_z(\gamma)=\left[\begin{matrix}cos\gamma&-sin\gamma&0\\sin\gamma&cos\gamma&0\\0&0&1\end{matrix}\right]$$

Finally, if we want to scale the vector in certain directions by some amount $S_x,S_y,S_z$ , we can construct a scaling matrix

$$S=\left[\begin{matrix}S_x&0&0\\0&S_y&0\\0&0&S_z\end{matrix}\right]$$

Therefore, if we want to scale a vector, then rotate, then translate, our final transformation matrix would be:

$$T=\left[\begin{matrix}RS&t\\0&1\end{matrix}\right]$$

projective transformations occur when the final row of T is not $\left[\begin{matrix}0&0&0&1\end{matrix}\right]$

Different Camera Models

In the weak perspective model, points are first projected to the reference plane using orthogonal projection and then projected to the image plane using a projective transformation.

Overall, weak perspective models result in much simpler math, at the cost of being somewhat imprecise. However, it often yields results that are very accurate when the object is small and distant from the camera.

Lecture

后记

母亲节快到了呢，还没有想好送什么

每次日子将近的时候，就会越来越纠结

然后 发现自己要干的事情还挺多的呢

加油啦

虽然最近因为夏令营九推的事情，没有什么经验

有点愁，但还是很认真地在做呢

还有两个大题啦！

转载请注明出处，谢谢。

愿 我是你的小太阳