mirror of
https://github.com/jackyzha0/quartz.git
synced 2026-03-24 15:05:42 -05:00
163 lines
4.8 KiB
Markdown
163 lines
4.8 KiB
Markdown
---
|
||
title: "10-3d-Cameras"
|
||
tags:
|
||
- lecture
|
||
- cosc342
|
||
---
|
||
|
||
|
||
CAMERAS AND PROJECTIONS
|
||
- Cameras project the 3D world onto a 2D image
|
||
- Input is 3D points: (𝑥, 𝑦, 𝑧)
|
||
- Output is 2D points: (𝑢, 𝑣)
|
||
|
||
- What form should P have?
|
||
> [!INFO] need to apply a transformation to convert 3d coords to 2d coords
|
||
> P should be a 3 row and 4 column matrix
|
||
|
||
WHICH CUBE LOOKS RIGHT?
|
||
|
||
> [!INFO] each cube is a projection of 3d points onto 2d space.
|
||
> middle cube is perspective transformation
|
||
> left is isometric
|
||
> right is orthographic
|
||
|
||
ORTHOGRAPHIC PROJECTION
|
||
- Simple way to go from 3D to 2D
|
||
- Delete one dimension!
|
||
- Deleting X projects to the X -Y plane
|
||
|
||
- This is not how our eyes work
|
||
> [!INFO] z coordinate is removed since the third column is zero
|
||
|
||
PERSPECTIVE PROJECTION
|
||
- Our view of the world:
|
||
- Distant objects looks smaller
|
||
- Parallel lines in 3D converge in 2D
|
||
- The pinhole camera
|
||
- A simple, but useful, model
|
||
- There is a central point of projection (the pinhole, often a lens in reality)
|
||
- Light travels from the world, through the pinhole, to the image plane
|
||
> [!INFO] need a hole that is big enough to get enough light
|
||
> but small enough to create a sharp image
|
||
> light goes through the hole to the image plane
|
||
> pin hole is also the "lens"
|
||
|
||
THE PINHOLE CAMERA MODEL
|
||
|
||
|
||
> [!INFO] use negative of f as it is behind the pinhole
|
||
> find U using similar triangles rule
|
||
|
||
|
||
> [!INFO] now we can project a point from 3d to 2d
|
||
> z is multiplies by 1 in the matrix so that the 3rd point of the homogenous coord becomes the z value
|
||
|
||
- We can put the image plane in front of the pinhole
|
||
- Removes the sign change
|
||
- Not practical for real cameras
|
||
- The maths works out just fine
|
||
|
||
|
||
|
||
> [!INFO] cant really convert from 2d back to 3d without knowing focal length and z coord of every point
|
||
|
||
TRANSFORMING CAMERAS
|
||
|
||
INTRINSICS AND EXTRINSICS
|
||
- Often break this down into
|
||
|
||
|
||
- Most simple case: 𝐮 = K[I 𝟎]<5D>
|
||
- K: camera calibration or intrinsics
|
||
- [I | 0]: camera pose or extrinsics
|
||
|
||
|
||
CAMERA CALIBRATION
|
||
|
||
- The model has image origin in the centre of the frame
|
||
- We usually put this at the top corner
|
||
- Can fix this with a translation
|
||
- If the centre is at $(c_u, c_v)$
|
||
|
||
|
||
|
||
|
||
TRANSFORMING CAMERAS
|
||
- We have assumed
|
||
- A camera at the origin
|
||
- Pointing along the +ve $Z$ axis
|
||
- We will need the general case
|
||
- Move the camera to any location
|
||
- Point the camera in any direction
|
||
|
||
> [!INFO] camera in games etc. always moves around with the player/operator,
|
||
> instead of transforming the camera we transform the world
|
||
> only need to apply inverse matrix to the 3d points of the world
|
||
|
||
TRANSFORM THE WORLD (!)
|
||
- To transform a camera by $T$
|
||
- Apply inverse, $T^{-1}$ , to points
|
||
- To move the camera left 3 units, move the world right 3 units
|
||
- To rotate the camera $45⁰$ about $Z$, rotate the world $-45⁰$ about Z
|
||
- The relative motion of the camera and the world is the same
|
||
|
||
ROTATE AND TRANSLATE A CAMERA
|
||
|
||
|
||
CAMERA CALIBRATION
|
||
```
|
||
double cv::calibrateCamera(
|
||
// Input parameters
|
||
//std::vector> objectPoints,
|
||
//std::vector> imagePoints,
|
||
//cv::Size imageSize,
|
||
// Output parameters
|
||
//cv::Mat cameraMatrix,
|
||
//std::vector distCoeffs,
|
||
//std::vector rotationVectors,
|
||
//std::vector translationVectors);
|
||
```
|
||
|
||
CALIBRATION TARGETS
|
||
- Input is 3D-2D matches
|
||
- Want easy-to find 2D points
|
||
- Need known 3D co-ordinates
|
||
- Planar targets common
|
||
- Easy to make with a printer
|
||
- Chess/Checkerboards
|
||
- Grids of dots or lines
|
||
- Is a 2D pattern enough?
|
||
|
||
|
||
3D POINT LOCATIONS
|
||
- Choice of co-ordinates
|
||
- X-Y plane is the target
|
||
- Origin at one internal corner
|
||
- Z runs into the target plane
|
||
- Need to decide on units
|
||
- Best to use real-world units
|
||
- Here 1 square = 1cm
|
||
- Can use 1 square = 1 unit
|
||
|
||
|
||
FINDING CHECKERBOARD CORNERS
|
||
- OpenCV method (in brief)
|
||
- Threshold image to B&W
|
||
- Look for quads
|
||
- Link quads -> checkerboard
|
||
- Sub-pixel refinement
|
||
|
||
|
||
MATCHING 2D TO 3D POINTS
|
||
- Corners are in rows/cols
|
||
- View of all corners required
|
||
- Aligns 2D corners to 3D target point locations
|
||
- Targets often odd-sized
|
||
- Example here is 7x6
|
||
- Why is this better than 8x8?
|
||
- Is this required?
|
||
|
||
|
||
|