VTuberWebcam

VTuber WebCam is an app for VTubers · By kitsune.ONE team

Add to collection

app Community Devlog

VTuberWebcam community

Alternative to realsense?

A topic by limejet created Aug 19, 2022 Views: 359 Replies: 4

Viewing posts 1 to 2

limejet2 years ago

I can see that you’re using pyrealsense2, which i assume is just a drop-in thing for realsense support. Considering that intel dropped the product line and motion tracking seems to no longer be officially supported, do you think it might be worth it looking into supporting alternatives?

As an example, i’ve got a generic usb camera that’s branded as “windows hello compatible”. There are a few of these now, all based on the tech in the kinect if i’m not mistaken. It’s got an additional ir camera in it for facial recognition through depth imaging, which seems to be the same tech that’s in the realsense cameras. The ir camera is directly accessible as a video device in linux. Supporting these cameras would open up 3d tracking to people who weren’t able to snag a realsense before they disappeared from the market.

I’ve not seen any drop-in replacements for pyrealsense2, but opencv has support for a variety of stereo camera setups so it may be possible.

kitsune.ONE teamDeveloper2 years ago(+1)

Intel RealSense definitely has alternatives. Depth cameras are heavily used in robotics, so not only vtubers are needed them xD
Possible alternatives: Orbbec, Asus Xtion, Kinect (not sure if it's still alive).

As an example, i’ve got a generic usb camera that’s branded as “windows hello compatible”.

I believe those are RGB-only with some extra sensors. I checked specs for the few of them. I need at least 1 RGB + 2 IR sensors in one single camera.

limejet2 years ago

That makes sense for stereo and all that.

I did some experiments and although this camera didn’t work with the realsense driver, it does do ir, and as far as i can tell it does it in much the same way as the original kinect (the one for the 360). There are a set of ir leds around the sensor, and when using the ir video device these leds strobe. This makes the camera capture a sort of black and white heightmap:

The framerate isn’t very good (a few Hz at most), and shiny things like glasses make the image all sorts of weird, but it definitely can do it, and it can do both an rgb stream and an ir stream at the same time.

With the right kind of opencv magic, this could definitely be something, but i get not wanting to spend time on it when there are actual depth cameras available.

kitsune.ONE teamDeveloper2 years ago

This is just an IR image. You will need 2 of them (left+right) for the stereo. And then you need to calculate and build the depth image. This part is done in camera's own SDK (for example RealSense, Leap Motion).

Found another possible alternative: MYNT EYE. Also some people are modded PS4 camera to make it work with OpenCV as depth camera.

limejet2 years ago

Okay! Thanks for at taking the time. Now i have some alternatives to learning opencv myself at least :)

itch.io

VTuberWebcam

Alternative to realsense?