Skip to main content

Camera Devices

What are Camera Devices?​

Camera Devices are the physical (or "virtual") devices that can be used to record videos or capture photos.

  • Physical: A physical Camera Device is a camera lens on your phone. Different physical Camera Devices have different specifications, such as different capture formats, resolutions, zoom levels, and more. Some phones have multiple physical Camera Devices. Examples:
    • "Backside Wide-Angle Camera"
    • "Frontside Wide-Angle Camera (FaceTime HD)"
    • "Ultra-Wide-Angle back camera"
  • Virtual: A virtual camera device is a combination of one or more physical camera devices, and provides features such as virtual-device-switchover while zooming (see video on the right) or combined photo delivery from all physical cameras to produce higher quality images. Examples:
    • "Triple-Camera"
    • "Dual-Wide-Angle Camera"

Select the default Camera​

If you simply want to use the default CameraDevice, you can just use whatever is available:

const device = useCameraDevice('back')

And VisionCamera will automatically find the best matching CameraDevice for you!

🚀 Continue with: Camera Lifecycle

Custom Device Selection​

For advanced use-cases, you might want to select a different CameraDevice for your app.

A CameraDevice consists of the following specifications:

  • id: A unique ID used to identify this Camera Device
  • position: The position of this Camera Device relative to the phone
    • back: The Camera Device is located on the back of the phone
    • front: The Camera Device is located on the front of the phone
    • external: The Camera Device is an external device. These devices can be either:
  • physicalDevices: The physical Camera Devices (lenses) this Camera Device consists of. This can either be one of these values ("physical" device) or any combination of these values ("virtual" device):
    • ultra-wide-angle-camera: The "fish-eye" camera for 0.5x zoom
    • wide-angle-camera: The "default" camera for 1x zoom
    • telephoto-camera: A zoomed-in camera for 3x zoom
  • sensorOrientation: The orientation of the Camera sensor/lens relative to the phone. Cameras are usually in landscape-left orientation, meaning they are rotated by 90°. This includes their resolutions, so a 4k format might be 3840x2160, not 2160x3840
  • minZoom: The minimum possible zoom factor for this Camera Device. If this is a multi-cam, this is the point where the device with the widest field of view is used (e.g. ultra-wide)
  • maxZoom: The maximum possible zoom factor for this Camera Device. If this is a multi-cam, this is the point where the device with the lowest field of view is used (e.g. telephoto)
  • neutralZoom: A value between minZoom and maxZoom where the "default" Camera Device is used (e.g. wide-angle). When using multi-cams, make sure to start off at this zoom level, so the user can optionally zoom out to the ultra-wide-angle Camera instead of already starting zoomed out
  • formats: The list of CameraDeviceFormats (See "Camera Formats") this Camera Device supports. A format specifies:

Examples on an iPhone​

Here's a list of some Camera Devices an iPhone 13 Pro has:

  • Back Wide Angle Camera 1x (['wide-angle-camera'])
  • Back Ultra-Wide Angle Camera 0.5x (['ultra-wide-angle-camera'])
  • Back Telephoto Camera 3x (['telephoto-camera'])
  • Back Dual Camera 1x + 3x (['wide-angle-camera', 'telephoto-camera'])
  • Back Dual-Wide Camera 0.5x + 1x (['ultra-wide-angle-camera', 'wide-angle-camera'])
  • Back Triple Camera 0.5x + 1x + 3x (['ultra-wide-angle-camera', 'wide-angle-camera', 'telephoto-camera'])
  • Back LiDAR Camera 1x (['wide-angle-camera'] + depth)
  • Front Wide Angle 1x (['wide-angle-camera'])
  • Front True-Depth 1x (['wide-angle-camera'] + depth)

Selecting Multi-Cams​

Multi-Cams are virtual Camera Devices that consist of more than one physical Camera Device. For example:

  • ultra-wide + wide + telephoto = "Triple-Camera"
  • ultra-wide + wide = "Dual-Wide-Camera"
  • wide + telephoto = "Dual-Camera"

Benefits of Multi-Cams:

  • Multi-Cams can smoothly switch between the physical Camera Devices (lenses) while zooming.
  • Multi-Cams can capture Frames from all physical Camera Devices at the same time and fuse them together to create higher-quality Photos.

Downsides of Multi-Cams:

  • The Camera takes longer to initialize and uses more resources

To use the "Triple-Camera" in your app, you can just search for a device that contains all three physical Camera Devices:

const device = useCameraDevice('back', {
  physicalDevices: [
    'ultra-wide-angle-camera',
    'wide-angle-camera',
    'telephoto-camera'
  ]
})

This will try to find a CameraDevice that consists of all three physical Camera Devices, or the next best match (e.g. "Dual-Camera", or just a single wide-angle-camera) if not found. With the "Triple-Camera", we can now zoom out to a wider field of view:

If you want to do the filtering/sorting fully yourself, you can also just get all devices, then implement your own filter:

const devices = useCameraDevices()
const device = useMemo(() => findBestDevice(devices), [devices])

Selecting external Cameras​

VisionCamera supports using external Camera Devices, such as:

Since external Camera Devices can be plugged in/out at any point, you need to make sure to listen for changes in the Camera Devices list when using external Cameras:

The hooks (useCameraDevice(..) and useCameraDevices()) already automatically listen for Camera Device changes!

const usbCamera = useCameraDevice('external')

🚀 Next section: Camera Lifecycle​