Unable to select HD capture resolution and high FPS (v2.0, beta 4)

[ertdfgcvb]

Testing the version 2.0 beta 4 library.

System: Windows 10 pro (64 bit)
Camera: Logitech HD Pro Webcam C920
Using GStreamer bundled with the library.

Currently testing the new beta with this pretty common webcam model and it works flawlessly with the default values
cam = new Capture(this, "HD Pro Webcam C920");
expect that I'm unable to select a higher resolution than (about) 640x480.
Even when passed the correct size and frame rate in the constructor
cam = new Capture(this, 1920, 1080, "HD Pro Webcam C920", 30);
the resulting image is just upscaled from 640x480.
When not passing the frame rate
cam = new Capture(this, 1920, 1080, "HD Pro Webcam C920");
the resolution is good but at ~5FPS.

As stated in many other issues apparently it's either FPS or resolution, never both.
When testing the camera with the bundled Logitech software I'm able to preview HD images at 30FPS so it's not a system bottleneck but must be Java or GStreamer related.

[ertdfgcvb]

Similar or duplicate of #92

[codeanticode]

I have not experienced those issues with similar HD webcams, would need to double check.

[cacheflowe]

Wow, yes this seems to be solvable with the custom gstreamer config noted in issue #92

I have a 1080p webcam running in Processing... it's beautiful 😍

[cacheflowe]

A note about this that I've been meaning to submit....

When using this method to get higher-resolution video streams, I experience very consistent crashing. If I use the default config (no special gstreamer args), everything is pretty stable. But if I access a 720p or 1080p video stream, it almost never runs for more than a couple of minutes. This is on Windows 11 with the latest Processing & Video libraries.

[codeanticode]

Ok, re-opening then! We have to explore what's going on with HD streams. I'm pretty sure that I have tested the library with HD webcams, but perhaps did not run the sketch long enough to notice the issue.

[cacheflowe]

I finally have more info on this error. And it could be somewhat related to issue #166

Usually I have no helpful error messages, but I finally got one. I also found a way to reliably trigger the crash. It seems especially likely to happen with a 1080p Capture stream, and if I resize the app window, it crashes right away. There's always an error coming from the NVIDIA nvoglv64.dll (I see this every time). And if I set the size() to 1920x1080, it crashes immediately and I get the Processing Texture error, which doesn't show at the smaller app size! If I leave it running after a successful start, it usually crashes within minutes, with the same NVIDIA native error.

import processing.video.*;
public Capture cam;

void setup() {
  size(1280, 720, P3D);

  String[] cameras = Capture.list();
  printArray(cameras);
  cam = new Capture(this, 1920, 1080, "pipeline: ksvideosrc device-index=1 ! image/jpeg, width=1920, height=1080, framerate=30/1 ! jpegdec ! videoconvert");
  cam.start();
  
  getSurface().setResizable(true);
}

void draw() {
  if(cam.available() == true) cam.read();
  image(cam, 0, 0, cam.width, cam.height);
}

Here's the stack trace, with the common nvoglv64.dll error, and the Texture error that only shows with a 1080p app size:

Processing video library using bundled GStreamer 1.20.3
Scanning GStreamer plugins... Done.
[0] "Integrated Camera #1"
[1] "USB Video Device"
#
# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffdb0282c70, pid=22364, tid=25984
#
# JRE version: OpenJDK Runtime Environment Temurin-17.0.6+10 (17.0.6+10) (build 17.0.6+10)
# Java VM: OpenJDK 64-Bit Server VM Temurin-17.0.6+10 (17.0.6+10, mixed mode, sharing, tiered, compressed oops, compressed class ptrs, g1 gc, windows-amd64)
# Problematic frame:
# C  [nvoglv64.dll+0xf22c70]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# An error report file with more information is saved as:
# C:\Users\cacheflowe\processing-4.2\hs_err_pid22364.log
java.lang.NullPointerException: Cannot invoke "java.util.LinkedList.add(Object)" because "this.usedBuffers" is null
	at processing.opengl.Texture.copyBufferFromSource(Texture.java:827)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at processing.video.Capture$NewSampleListener.newSample(Unknown Source)
	at org.freedesktop.gstreamer.elements.AppSink$2.callback(AppSink.java:232)
	at jdk.internal.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.invokeCallback(CallbackReference.java:585)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.callback(CallbackReference.java:616)
java.lang.NullPointerException: Cannot invoke "java.util.LinkedList.add(Object)" because "this.usedBuffers" is null
	at processing.opengl.Texture.copyBufferFromSource(Texture.java:827)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at processing.video.Capture$NewSampleListener.newSample(Unknown Source)
	at org.freedesktop.gstreamer.elements.AppSink$2.callback(AppSink.java:232)
	at jdk.internal.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.invokeCallback(CallbackReference.java:585)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.callback(CallbackReference.java:616)
java.lang.NullPointerException: Cannot invoke "java.util.LinkedList.add(Object)" because "this.usedBuffers" is null
	at processing.opengl.Texture.copyBufferFromSource(Texture.java:827)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at processing.video.Capture$NewSampleListener.newSample(Unknown Source)
	at org.freedesktop.gstreamer.elements.AppSink$2.callback(AppSink.java:232)
	at jdk.internal.reflect.GeneratedMethodAccessor5.invoke(Unknown Source)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:568)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.invokeCallback(CallbackReference.java:585)
	at com.sun.jna.CallbackReference$DefaultCallbackProxy.callback(CallbackReference.java:616)
#
# If you would like to submit a bug report, please visit:
#   https://github.com/adoptium/adoptium-support/issues
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.
#
Could not run the sketch (Target VM failed to initialize).
For more information, read Help ? Troubleshooting.

[ColtonCreations]

void setup() { size(1280, 720, P3D); // here you set the sketch to run at 720p
cam = new Capture(this, 1920, 1080, // here you try to capture from a cam at higher resolution than the sketch was set for

try this:
void setup(){ fullScreen(P3D); }

or
void settings(){ fullScreen(P3D); }
if you are using Processing version that includes settings()