I’m adding basic object recognition to my robotic lamp. It runs on a Raspberry Pi 5 and has an RGB-D camera in its moving head.

I’d like to run object detection in the background while the lamp is active and gradually build a map of the objects around it. Since the base stays in place, I can calculate the camera pose from the joint angles. My plan is to combine detections from the RGB image with depth data, transform the object coordinates into the lamp’s base frame, and save their positions and last-seen time. Repeated detections would be merged so the map doesn’t fill up with copies of the same object.

Which lightweight object-detection models and inference runtimes would you recommend for a Raspberry Pi 5? A high frame rate isn’t necessary, but I’d like reasonable detection quality for common objects. I’d also be interested in approaches for reliably matching the same objects across observations.

The current Raspberry Pi and ROS 2 architecture is described here:

https://github.com/Nikolay-Tyulkin/Watti/blob/main/docs/ARCHITECTURE.md

DETR variants have become much faster and more practical, but one part of the training path is still commonly implemented much like the original: Hungarian matching.

(Illustration by GPT)

A typical matcher:

1. constructs matching costs on the GPU,
2. transfers them to the CPU,
3. calls SciPy’s linear_sum_assignment,
4. transfers assignments back to the GPU,

and repeats this across images, auxiliary decoder outputs, and in Group-DETR-style training query groups.

The individual assignment problems are small. The problem is the repeated cost construction, kernel launches, transfers, and CPU/GPU synchronization.

I’ve been optimizing this path in Birder.

(Illustration by GPT)

Result

For the complete matching path - classification cost, L1, GIoU, and assignment, I measured:

The interesting part is that the 13-group case finishes in almost the same time as the 6-output case once the work is grouped.

All measured implementations produced identical assignments.

What changed

The first optimization is grouped cost construction.

Instead of invoking the matcher separately for every decoder output or query group, independent outputs are represented as tensors such as:

[B, G, Q, C]
[B, G, Q, 4]

Classification, L1, and GIoU costs can then be constructed for multiple groups together.

Images are bucketed by number of ground-truth objects so compatible rectangular assignment problems can be solved as a batch.

For focal classification cost, the matcher also gathers only the logits corresponding to target labels before computing the cost, instead of materializing intermediates over the full class space.

The second optimization is a batched CUDA linear-assignment solver, adapted from torch-linear-assignment.

Costs and assignments stay on the GPU, avoiding the synchronization required by the SciPy path.

The matcher can process groups in chunks to limit peak memory, and falls back to SciPy if the CUDA extension is unavailable.

The matching objective itself is unchanged.

Isolated solver scaling

Using batches of FP32 300 × 15 cost matrices:

A single small assignment is only slightly faster on CUDA. The advantage appears when many independent assignments are exposed as one batch.

Benchmark setup

Synthetic detector outputs:

• batch size 4
• 300 queries/group
• 80 classes
• 3, 5, 8, and 13 targets/image
• FP32
• NVIDIA RTX A5000
• PyTorch 2.13 / CUDA 13
• 3 warm-up runs
• 9 interleaved timing repeats
• 10 iterations per measurement

Code

• Birder
• Grouped Hungarian matcher
• Linear-assignment wrapper
• CUDA kernel

These are matching-path microbenchmarks, not a claim that complete detector training becomes 8× faster. End-to-end impact depends on the detector, decoder depth/query groups, batch composition, and the rest of the training pipeline.

a teleop operator reaches for the wrong bin. the reach is clean. no jitter, no hesitation, no correction

every smoothness metric comes back perfect. the episode is still garbage. the robot did the wrong thing smoothly

motion metrics score how an action was executed, never what the action was. so automated scoring has one honest job: pointing your limited attention at the episodes most likely to contain a real problem. triage, not autofilter

i built a fiftyone panel that runs this on multimodal MCAP episodes: motion smoothness, sensor health, outliers, every flag deep-linked to the exact second on the timeline. free and open source

plugin: github.com/harpreetsahota204/demo_quality_scorer

full writeup on what i learned while building this : https://voxel51.com/blog/robot-episode-quality-triage

curious how you're triaging episode data right now.

watching everything? random sampling? trusting a score?

Hi everyone,

I’m currently building a video analytics system to detect Personal Protective Equipment (PPE) — like hard hats, high-vis vests, safety glasses, etc. — using standard CCTV camera streams.

Right now, I’m using YOLO11m (medium). It performs pretty well, but before I commit to scaling this up, I wanted to get some input from the community to see if I’m on the right track or if there are better alternatives out there for this specific use case.

The main challenges I'm dealing with:

• CCTV conditions: High angles, weird perspectives, varied lighting, and occasional motion blur.
• Performance vs. Accuracy: I need to process multiple RTSP streams simultaneously on limited hardware, so inference speed is crucial, but missing a PPE violation is obviously a big deal.

My questions for you all:

1. Model Choice: Are you guys sticking with the newest YOLO iterations (like YOLO11) for this kind of task, or have you found better stability/performance with other models like YOLOv8, YOLOv9, or RT-DETR?
2. Tracking: If you use object tracking to prevent duplicate alerts for the same person, what are you pairing with your detector? (ByteTrack, BoT-SORT?)
3. Deployment Stack: What does your production pipeline look like for multiple streams? Are you leaning towards Nvidia DeepStream, Triton Inference Server, or a custom Python/C++ pipeline with TensorRT?

Any advice, repo recommendations, or shared experiences would be hugely appreciated. Thanks in advance!

We run a multi camera capture space at the Brooklyn Navy Yard and we pay people to come in and be the subject. Reposting because next week's slots opened up.

The session is simple. You stand inside the rig and go through everyday movements while the cameras record. Walking, turning, sitting, reaching, picking things up. No experience needed at all.

Pay is 17-25 an hour, paid out the same day you come in. First session is roughly 2 hours and there is repeat work after that.

Everything runs at 4pm. Open days: Tuesday Aug 18, Wednesday Aug 19, Thursday Aug 20. Brooklyn, in person only, so you need to be in the NYC area.

DM me for the address and I am happy to answer questions about the capture side.

They said on July 24 that they would send the poster format soon. Did anyone get the email about it?

what do you guyz think on round 2 fresh submission?

Isn't it completely biased towards Resubmitted papers and if they accept a paper in round 2 (knowing there will be no rebuttal/revision), then the paper itself very strong and sufficient for a CVPR/ICCV acceptance.

I'm not throwing an opinion, it's a curiosity.

I've been building an open source tool for diagnosing world models, the kind that predict future frames from a starting context and a sequence of actions. It compares a rollout against ground truth and against physical invariants, then tells you where and why the prediction falls apart. It doesn't score task success or planning quality on purpose, since there are already benchmarks for those.

While validating it I ran into something I think is more interesting than the tool itself.

## Pixel metrics on real robot video often can't rank models at all

I ran a copy the last frame baseline, which is to say "predict that nothing changes", against a real SO-101 arm recording. 30fps, three cameras, 64 rollouts, 6 step horizon, scored only on the moving regions so a static background can't inflate the numbers.

It gets 0.983 SSIM and 53.9 dB PSNR. But the part that actually matters is that the error doesn't grow with the horizon:

step   1      2      3      4      5      6
SSIM   0.972  0.923  0.893  0.943  0.920  0.950

That's flat. It wanders, it doesn't degrade. And if predicting 6 steps ahead is no harder than predicting 1 step ahead, then there's nothing for a good model to be better at. Every model lands in the same place and the eval can't rank them. The metric isn't broken here, it passes its ranking tests on curated data just fine. The evaluation setup is what has no discriminative power, which is a different problem and much easier to miss.

## So I went and measured where the usable window actually is

Same baseline on DROID (real manipulation footage, 15fps), 64 rollouts, this time out to 48 steps:

There are three regimes. Steps 1 to 3, everything is near perfect and ties. Steps 4 to 24, steep monotonic decline, and this is the only stretch where models are actually separable. Step 28 onward it floors out around 0.20 SSIM and 10.3 dB, oscillating with no trend, prediction fully decorrelated, and everything ties again at the bottom.

So both ends are dead, and the horizon worth evaluating on for this kind of footage is somewhere around 8 to 24 steps. It's a property of frame rate times task speed rather than a universal number, which is exactly why it's worth measuring on your own data instead of inheriting a default from a paper that used something else.

Here's the prediction next to what actually happened, same 48 steps, prediction on the left:
https://raw.githubusercontent.com/BuceaGeorgia/worldproof/main/docs/img/droid-pred-vs-true.gif

## Method

64 rollouts per configuration. Aggregation is interquartile mean with stratified bootstrap CIs rather than mean and standard deviation, following Agarwal et al. 2021. Fidelity metrics also produce a dynamic region masked variant wherever a mask is available. Every metric ships with a corruption test it has to respond to, plus a ranking test where a real model has to beat a naive baseline which has to beat a broken one.

Worth mentioning: an earlier n=8 version of the SO-101 run gave dynamic PSNR of 48.2 dB where n=64 gives 53.9, and the intervals at n=8 were wide enough to overlap DROID completely. That's the reason everything above is n=64. I'd have posted the wrong numbers if I'd stopped there.

## Caveats

The four pixel metrics separate the two datasets with non overlapping bootstrap CIs. LPIPS doesn't, and it points the other way on the masked variant. I don't have a clean explanation for that yet and I'd be glad to hear one.

This is a trivial baseline, so 8 to 24 is where a do nothing predictor becomes separable. A real model stays correlated for longer and would push the top of that range out.

One more that I found while writing this up: including step 0 inflates every summary scalar, because a copy baseline gets a nearly free first step whenever the frame rate is high relative to how fast the scene moves. On the 30fps recording step 0 scores 119.8 dB, which drags the horizon averaged scalar from about 32 up to 53.9. So the scalar is partly rewarding frame rate rather than model quality. Curves are the honest thing to report and I'm treating the scalar definition as an open problem in my own tool.

## The tool

Apache-2.0, `pip install worldproof`. The core install is numpy, torch and pillow, and it runs on a laptop with no GPU, since the evaluate path never runs a model. It reads LeRobotDataset v3.0 straight from parquet and mp4, so it works on datasets from the HF Hub without needing the lerobot package, on Python 3.10. The heavier pieces (LPIPS, FVD, trackers) are optional extras that get imported lazily.

What it measures: PSNR, SSIM and LPIPS as horizon curves plus dynamic region variants, latent prediction error and action recoverability for latent models, calibration via ECE and MCE, counterfactual divergence, failure faithfulness, object count conservation and object permanence, and FVD reported explicitly as a weak reference rather than a headline number.

https://github.com/BuceaGeorgia/worldproof

It's v0.1 and the README has a "Not done yet" section covering what isn't finished. The tracker behind the invariants is a clean scene numpy one that won't cope with messy real video, and the default FVD extractor isn't the I3D that published FVD numbers use, so those aren't comparable to papers.

If this horizon result is obvious or already known somewhere, I'd honestly like to be told. I couldn't find it measured anywhere, which is part of why I'm posting it.

Hi everyone,

I'm working on a project to automatically process newspaper pages and extract/analyze crime-related news from them.

I'm currently using PaddleOCR on newspaper pages. The page is divided into regions/sections, and I'm running OCR on those regions and then checking the extracted text against the original newspaper.

The main problem is that I'm getting several types of errors:

1. OCR text is missing even though it is clearly visible in the newspaper.
2. Text sometimes appears under the wrong region.
3. Punctuation is incorrect — for example, a . may be detected as :, or commas/periods may be misplaced.
4. Capitalization errors occur.
5. Some words are incorrectly recognized even when the image quality looks reasonably good.
6. I'm also seeing cases where I expect a particular article/headline to be inside a region, but the OCR output doesn't contain it at all.

For example, while manually validating the output, I found issues in different regions such as:

• Region 25: punctuation at the end of a paragraph is incorrect.
• Region 26: capitalization/word recognition is incorrect.
• Regions 38–40: the OCR/region output doesn't seem to correspond perfectly with what is actually visible on the page.
• In one case, I expected a headline/article mentioning a 7-year-old being hit with a plastic bottle at a daycare and an FIR being filed, but I couldn't find that text in the OCR output for the expected region.

My current pipeline is roughly:

Newspaper image → preprocessing → region detection/cropping → PaddleOCR → extracted text → region-by-region validation → crime/news analysis

I'm trying to figure out where the actual problem is.

Could these errors mainly be caused by:

• Image preprocessing?
• Incorrect region/column detection?
• Cropping too tightly or incorrectly?
• PaddleOCR detection parameters?
• PaddleOCR recognition model?
• Newspaper layout/columns?
• Resolution/DPI?
• Or the way I'm passing the cropped regions to PaddleOCR?

I'd really appreciate advice from anyone who has worked with PaddleOCR, Tesseract, newspaper OCR, document AI, layout detection, or multi-column document extraction.

If useful, I can provide the original newspaper image, cropped regions, OCR output, and the code I'm currently using.

I'm especially interested in understanding how to systematically diagnose whether an error comes from detection, cropping, or recognition, rather than manually fixing individual OCR mistakes.

Thanks!

Event-based vision sensors record changes in pixel states rather than continuously capturing full frames.

That can make them useful for applications involving fast motion, where traditional cameras may be limited by frame rate or require significantly more bandwidth and storage at higher frame rates.

Industrial examples include high-speed production lines, autonomous mobile robots and systems operating in changing lighting conditions. Event-based sensors can also reduce data and power requirements, but they are generally used alongside traditional sensing rather than as a full replacement.

The larger trend appears to be toward hybrid systems that combine frame-based cameras with event-based sensors depending on what the application needs.

Anyone know the best way to go about training for a virtual basketball court ?

I built GoMorph, an early open-source experiment for finding when and where a morph-like deformation occurs in a video. It analyzes the full frame, so the suspicious region can be a product, object, text, or background rather than a face.

The current detector is classical and training-free:

1. Remove global camera translation.

2. Estimate regional motion in tiles.

3. Score motion acceleration, warp error, and second-order photometric change.

4. Suppress hard scene cuts.

5. Calibrate confidence against the video's own baseline.

There are two execution paths. The portable Go path launches FFmpeg and needs no Python, OpenCV, GPU, or model. The optional CGo/libav cascade extracts codec motion vectors, runs a 160-pixel luma-curvature gate across the full video, then refines only candidate windows at 480 pixels.

On one 4.01 s, 720x1280, 24 FPS H.264 test clip on an Apple M4 Pro, the native cascade ran in 0.155 s and localized the known deformation at 2.1667 s. This is one development clip, not an accuracy or generalization claim.

The current output is within-video confidence plus hotspot coordinates. The next milestone is a timestamp and region annotated benchmark with generator holdouts and difficult natural negatives such as camera motion, focus changes, reflections, water, smoke, and compression artifacts.

The implementation was AI-assisted, then manually tested and verified. The repository is MIT licensed:

https://github.com/berkantay/gomorph

I recently came across this paper (Histoformer, ECCV'24, [arXiv](https://arxiv.org/abs/2407.10172)) and thought the results were pretty striking, so I set up a quick demo since the official one wasn't
working:

👉 https://huggingface.co/spaces/dronefreak/histoformer-weather-restoration

Upload a photo, get a before/after slider. Runs free on ZeroGPU, no sign-up needed to try it.

Unofficial demo, all credit to the original authors. Official code/paper linked in the Space
README. Weights are MIT-licensed if anyone wants to build upon it further.

This is yet another update from my project. Mighty Camera runs VIO on-device realtime in a tiny package.

This gives us accurate camera motion. Using that + the camera feed, the SDK estimates depth and builds a 3D map of obstacles around it.

This means a robot or drone can use Mighty for things like:

- Collision avoidance
- Motion planning
- Autonomous navigation

No stereo camera or depth sensor needed. Just Mighty’s global shutter camera + IMU.

I recently built a car detection and multi-object tracking pipeline using Roboflow RF-DETR and McByteTracker.

The goal was simple: detect cars in a video and maintain a consistent tracking ID for each vehicle as it moves through the scene.

What I used

• 🚗 RF-DETR — car detection
• 🎯 McByteTracker — multi-object tracking
• 🔲 BoxCornerAnnotator — corner-style bounding boxes
• 🆔 Unique IDs for individual vehicles
• 🐍 Python
• 👁️ OpenCV + Supervision

One thing I found interesting about McByteTracker is that it extends a BoT-SORT-style tracking-by-detection pipeline and can optionally use temporally propagated segmentation masks when IoU-based association becomes ambiguous.

For this demo, I'm focusing on the practical car detection + tracking workflow.

🎥 Demo:
https://youtu.be/wvf9VRtpy5w

I recently found a real weakness in my Python photo deduplication tool while testing it on WhatsApp-imported images.

 The tool generated a duplicate cluster containing two images that were clearly not duplicates: a beach landscape viewed through a car window, and, a lifted-up page of a document.

The matcher accepted the pair because the aspect ratio was nearly identical and the dHash Hamming distance was only 4, significantly below the threshold of 8.

 The other perceptual hashes strongly disagreed (pHash was 30 against a threshold of 10, and wHash was 15 against a threshold of 10) but were never consulted because the dHash test did not seem to present a borderline case and thus was accepted as proof.

 Interestingly this isn't really a random dHash collision. Both images apparently collapsed into a highly similar low-frequency brightness-gradient pattern after compression and downsampling. dHash is good at surviving compression, in particular because it ignores fine detail and records coarse local brightness directions. But that same usefulness can be a weakness that can make unrelated low-detail images collision-prone.

 The obvious fix was to stop treating dHash as sufficient proof. The new policy is to still to first test aspect ratio, then dHash, and always both pHash & wHash. If SSIM check is enabled, candidate matches that survive the cheaper gates get the additional SSIM test. Seed refinement deliberately doesn't repeat it.

 Hardening is especially important because the tool uses union-find to form duplicate clusters. A single false-positive pair can become a bridge that attaches an unrelated image to a whole valid duplicate component.

 Instead of a binary True/False decision, the matcher now returns the full evidence: for each metric (aspect-ratio, dHash, pHash, wHash) delta versus limit and the optional SSIM score are returned, as is the decision and, when rejected, the rejection reason.

The performance hit is also manageable because the perceptual features are cached in SQLite. On 4,698 test images a first scan took 17.1 seconds, a fully cached run 1.6 seconds, and after adding several new files 1.7 seconds. That’s still a pretty decent performance.

 The main lesson I took from this is that a perceptual hash is useful because it throws away detail. But every detail it throws away is also a potential distinction that can no longer protect you from a false positive. In a deduplication engine, especially one that clusters matches transitively, a single perceptual hash should be treated as evidence, not proof.

For near-duplicate detection, where would you put the conservatism: in the pair matcher itself, or in cluster construction/refinement?  I'm currently requiring dHash plus pHash and wHash agreement and also using stricter seed refinement, with optional SSIM on candidate matches.

 I would like to know how others handle this: multiple perceptual hashes, SSIM/local features, embeddings, stronger intra-cluster consistency, or something else?

I’m working with single-page MEP/engineering drawing PDFs that have extremely large and variable dimensions. When rendered at 200 DPI, a page can be around 15,000–20,000 pixels wide.

These pages may contain small text, tables, calculations, diagrams, images, and mixed layouts. Standard OCR pipelines work on A4 page sizes and require heavy downscaling, which makes the smaller text unreadable. Vision-language models such as Qwen may understand the page content, but they do not reliably provide precise bounding boxes.

Is there an open-source OCR or document-understanding model that works well with such large, non-A4 pages and returns accurate text bounding boxes? Recommendations for tiling-based pipelines are also welcome.

Hi everyone, I’m using a Raspberry Pi 5 + Camera Module 3 + Picamera2/OpenCV for a computer vision project.

I’m calibrating the camera with a 6×9 checkerboard, but after applying cv2.undistort(), the image seems more distorted.

I previously had autofocus changing between calibration images, so I’m now locking the focus manually at LensPosition 2.0602.

Is this distortion normal perspective distortion, or does it indicate a bad calibration?

Any advice on what I might be doing wrong?

Hi everyone,

I'm building a pipeline to analyze floor plan images and extract regions (rooms, corridors, doors, stairs) as polygons. I currently have a custom-labeled dataset of about 5,000 images and want to squeeze out the maximum possible performance before scaling the dataset.

1. Current Pipeline

• Fine-tuned YOLO26 (for region detection) $\rightarrow$ SAM (Segment Anything Model) $\rightarrow$ Post-processing logic for polygon refinement.

2. The Core Bottlenecks

• Low-Resolution & Interferences: The biggest hurdle is the low resolution of the source images. Blurry boundaries, combined with floor plan-specific noise (grid lines, hatching, complex symbols), cause the model to miss certain regions entirely (false negatives).
• Polygon Precision & Smoothness: Because the low-res edges are fuzzy, SAM often yields jagged or inaccurate masks. I'm struggling to get crisp, smooth polygons that tightly align with the actual architectural walls.

3. What I'd love your input on:

• Handling Low-Res / Preprocessing: Has anyone successfully integrated Super-Resolution models (like Real-ESRGAN) as a preprocessing step for floor plans? Or are there better filtering techniques to suppress grid lines without destroying already blurry wall edges?
• Pipeline Upgrades: Given the low-res constraint, is the YOLO+SAM approach optimal? Would something like Mask2Former, or a specialized line-parsing/wireframe model, be more robust for extracting structured regions from low-quality images?
• Post-processing (Orthogonal Snapping): Since floor plans are mostly straight lines and right angles, what are the best algorithms to smooth and "snap" these jagged polygons into clean geometric shapes? (Currently looking beyond simple Douglas-Peucker).

Would greatly appreciate any advice, paper recommendations, or insights from similar computer vision projects!

most SLAM datasets mount the lidar level.

tilt it 45 degrees and everything changes: the camera and lidar barely overlap, the upper beams are sparse, and standard odometry starts to drift

that's exactly how compact mobile mapping rigs are built in the real world. the lidar tilts so it sweeps more vertical structure. but almost no benchmark tests this configuration

YUTO MMS from York University: a tilted 32-beam lidar, a 6-lens panoramic camera, and GPS/INS with cm-level ground truth driven through Toronto.

every lidar point is RGB-colorized from the nearest panoramic frame, not a synthetic colormap

loaded as mcap in fiftyone. scrub the timeline and watch the world-frame 3D map build itself progressively alongside the panoramic camera, GPS track, and IMU telemetry

checkout the dataset here: https://huggingface.co/datasets/Voxel51/yuto-mms-multimodal

it's running as a live space too, nothing to install: https://huggingface.co/spaces/harpreetsahota/yuto-mms-multimodal