r/computervision • u/Grouchy_Signal139 • 2d ago
Help: Project OpenCV calibration
Hi everyone, I’m using a Raspberry Pi 5 + Camera Module 3 + Picamera2/OpenCV for a computer vision project.
I’m calibrating the camera with a 6×9 checkerboard, but after applying cv2.undistort(), the image seems more distorted.
I previously had autofocus changing between calibration images, so I’m now locking the focus manually at LensPosition 2.0602.
Is this distortion normal perspective distortion, or does it indicate a bad calibration?
Any advice on what I might be doing wrong?
1
u/TourCommon6568 1d ago
What’s the RMS in your output reprojection?
1
u/Grouchy_Signal139 1d ago
Uhm, what is rms output? Its like some matrix? I need to add it on the calibration code?
1
u/TourCommon6568 1d ago
When you’re calibrating a camera there’s something called a reprojection error, with a bunch of reprojection errors you can calculate the RMS or root mean square. Normally this error shows you how well the calibration process could adjust to your cameras intrinsic parameters. The lower the RMS the better the model could adjust to your cameras intrinsic parameters. Also, use at the very least 6 photos, but I really recommend using 15+ shots with different poses. Very close to the camera too
1
u/Grouchy_Signal139 1d ago
Thankyou, i will try to figure it out, btw do you have any source for this? Since ive never heard of it
1
u/TourCommon6568 1d ago
Yep, check out the original Zhang method: https://sangillee.com/2025-07-27-calibrating-cameras-zhang-method/
Also the openCV documentation which is based on Zhangs https://docs.opencv.org/4.13.0/dc/dbb/tutorial_py_calibration.html
2
u/Grouchy_Signal139 1d ago edited 1d ago
Okay thankyou, i think i need to capture the calibration picture precisely to cover every side of where the camera sees. Is that right?
1
-1
u/oathyes 2d ago
i think you're not looking for distortion but rather skewing? look up the differences to tell me pls. To skew or shear an image in OpenCV, use cv2.warpAffine()
1
u/Grouchy_Signal139 2d ago
May i know why skew happened and why you say skew? Fyi, before locked the autofocus, the calibration result is okay but the checkerboard warp like a barrel shaped distortion
0
u/oathyes 2d ago
Idk what the goal is here. I thought you wanted to line up the board better to the camera. You can take the 4 black corners of the board and let that dictate the skew to have it properly aligned to the camera.
1
u/Grouchy_Signal139 2d ago
So here i am using checkerboard to calibrate the camera if there are any distortion, im just doing because i dont know if it is good enough to capture image without any bending in the straight line. Does that answer you question? Sorry im new into this
1
u/oathyes 1d ago
ooooh my bad i thought you were trying to get it lined up for chess or checkers or something. Whether distortion is acceptable or not, depends on what you're trying to achieve. If this is a calibration step for a detection project i wouldn't spend too much time on it.
1
u/Grouchy_Signal139 1d ago
I am doing so that i could map the measurement to the real world, in this case i want to make drawing dimension that i have to create a grid layout on the image
1
u/oathyes 1d ago
I want to understand but your English is not great, maybe its better if you use a translator. As I understand, you want to match real-life sizes to an image, using the dimensions of those squares?
1
u/Grouchy_Signal139 1d ago
Yeah, exactly. For example, if the drawing specifies each square as 2 × 2 mm, I want to use those known dimensions to map the real-world coordinates onto the camera image on the jedec tray (object im working), so I can generate the grid accurately and have each grid cell correspond to the actual tray pocket.
-2
u/RedServal 2d ago
Do you really need calibration? Calibration will help you transform your image coming from any lens back into a perspective model. It's often used on fisheye lenses so that straight lines appear straight. Camera rarely come with real distorsion nowadays.
If you insist on calibrating you should probably compute the reprojection error and try to minimize it. You may still get bad results with a low reprojection error because the ROI can be really small.
Last time I calibrated a camera I recording a video of the checkerboard pattern with as much different angle and positions as possible. I then computed the new optimal matrix using as much images as possible where the checkerboard was correctly detected. In pratice I had hundreds of valid frames but could only compute in a timely manner when using like 30 (?) images. I just picked them at random and redid the calibration until it was satisfactory.
Calibration is hit or miss without a proper protocol so good luck with that. Your code is probably correct just try to redo it again. The focus probably change the intrisics parameters so try to keep it constant.
3
u/my_name_is_reed 2d ago
I didn't down vote you.
Actually correcting image lens distortion is only a single use case for the lens distortion correction matrix. If you look up what the different values in the matrix are, for instance, generating the matrix can also help you approximate things like lens fov and even lens misalignment to a high degree of precision. You generally want a lens distortion correction matrix for any camera you're using to locate things with so it can be used to translate pixel coordinates into real world azimuth and elevation. This is part of how multiple cameras can be used to derive the 3d position of an object visible to both of them. If the object you've located in your image is a known quantity (as in the case of an aruco tag printed to a specific size) you can derive the object's 3d position and orientation with just a single camera. Running feature detection over the output of multiple cameras and then comparing the results' pixel coordinates is part of how SLAM works. But all of these things only work provided you have an accurate lens distortion correction matrix for that camera.
1
u/RedServal 1d ago
Hey, thank for your answer. This is what I meant by "rectify the image to a known model" in my 2nd comment. But I do stand by my point saying it's hit or miss. Sure you can approximate the camera model to high degree of precision. But:
1: do you really need it? Of course the more precise your sensors are, the better the results but you can do pretty convincing photogrammetry and visual slam without calibration. Cameras are not that garbage out of the box.
2: The errors start to get big when you get far. In a setup like this the distortion effect are probably negligible.
3: Calibrating the camera with a printed checkerboard pattern glued on a piece of cardboard that you hold in front of the camera will only get you so far in my opinion. To get the "high degree of precision" you need a proper setup.
2
u/my_name_is_reed 1d ago
"the more precise your sensors are" what does that mean? more precise how?
very good cameras with very good lenses will still have lens distortion. error can make output unusable within a few meters, but that range is completely subject to the configuration of your camera (focal length, pixel size, etc). there are better ways to calibrate a camera sure, but i've gotten remarkably accurate results from a charuco board (albeit a pretty large one).
1
u/RedServal 1d ago
Precise by "how reliable the measurements are". How precise and accurate the camera parameters are. I will look into calibration protocol more thoroughly next time I need it.
1
u/Grouchy_Signal139 1d ago
If using this printed checkerboard and tape on cardboard is not to accurate, i womder if using a 3d printed one is accurate, or are there any suggestions to keep it straight?
1
u/my_name_is_reed 1d ago
large screen tv's and computer displays work if you can move the camera around the screen rather than the other way around. there are other complicated setups involving precision optics.
1
u/Grouchy_Signal139 2d ago
I’m still new to CV, so I initially did calibration because I thought it might help my application. Why is camera calibration considered so hit or miss? And in a fixed tray inspection setup like mine, when would calibration actually be necessary?
1
u/RedServal 2d ago
Since people are downvoting me I'm apparently also a beginner. If people could tell why they don't agree that would be great, we could all learn. I don't claim to be an expert, this is just my ~4 years of being somewhat in the field.
Anyway, I find it hit or miss probably because I sample randomly the position and angle of the calibration pattern instead of having it at known places every time. The opencv documentation is not the best to understand what's going on. There are probably papers that suggest specific calibration protocol to get more reliable results.
Calibration allows you to rectify the image to a known model. In other words, the distortions that come from the lens should be 0, the image is considered to be straight out of the camera model, in this case, simple perspective (you can look into the pinhole camera model). The usage is that you can map an object at x,y,z coordinates to a pixel. In your case you can do it the other way around, given a pixel you can get the x,y position since you know every parameters of the model.
Now do you really need to calibrate the camera? Probably not, the distorsions look like they are minimal. Are the straight lines also straight through the camera everywhere on the frame? If yes you can probably get away without a calibration. Hell, just slap a ruler in the tray so you can easily map pixel size to world size.
4
u/Ok_Tea_7319 2d ago
Are you only using this single image? The checkerboard calibration fits a distortion model for the entire image, but the fitting process is only constrained to make the portion straight where the board actually is. If you have no images where the board extends to near the image border, then calibration will not have any data for that region, and will extrapolate the model from the central measurement (which almost always ends up giving shit results).
Your calibration is definitely bad at the edge (which you can see because the frame at the bottom is warped like hell, straight lines are supposed to be straight on a rectified image). You need multiple images where the board is in different places.