You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Recently I added a skipped frame counter to my Web implementation of Scrcpy client. Because rendering images on Web is slow, I have VSync on to avoid unnecessary works.
Then I found that the number of skipped frames is insanely high. On my Mi 11 with a 120FPS display and my PC monitor running at 144Hz, usually only about 70 frames are rendered, and the other 50 skipped.
In my further investigation, I found that the time those frames arrive are inconsistent. It's almost like every two frames are sent together. Here is part of my log:
So, I also tested with the official client, the results are same. I modified server code to output time took for each frame to encode, video output is also completely disabled to minimize interference.
diff --git a/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java b/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java
index e95896d3..90e48d0c 100644
--- a/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java+++ b/server/src/main/java/com/genymobile/scrcpy/ScreenEncoder.java@@ -9,6 +9,7 @@ import android.media.MediaCodecList;
import android.media.MediaFormat;
import android.os.Build;
import android.os.IBinder;
+import android.os.SystemClock;
import android.view.Surface;
import java.io.FileDescriptor;
@@ -42,6 +43,7 @@ public class ScreenEncoder implements Device.RotationListener {
private final boolean downsizeOnError;
private long ptsOrigin;
+ private long prevTime = 0;
private boolean firstFrameSent;
public ScreenEncoder(boolean sendFrameMeta, int bitRate, int maxFps, List<CodecOption> codecOptions, String encoderName,
@@ -146,6 +148,7 @@ public class ScreenEncoder implements Device.RotationListener {
private boolean encode(MediaCodec codec, FileDescriptor fd) throws IOException {
boolean eof = false;
+ prevTime = SystemClock.uptimeMillis();
MediaCodec.BufferInfo bufferInfo = new MediaCodec.BufferInfo();
while (!consumeRotationChange() && !eof) {
@@ -157,17 +160,21 @@ public class ScreenEncoder implements Device.RotationListener {
break;
}
if (outputBufferId >= 0) {
- ByteBuffer codecBuffer = codec.getOutputBuffer(outputBufferId);+ long now = SystemClock.uptimeMillis();+ Ln.i("Frame interval: " + String.valueOf(now - prevTime));+ prevTime = now;- if (sendFrameMeta) {- writeFrameMeta(fd, bufferInfo, codecBuffer.remaining());- }+ // ByteBuffer codecBuffer = codec.getOutputBuffer(outputBufferId);- IO.writeFully(fd, codecBuffer);- if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) == 0) {- // If this is not a config packet, then it contains a frame- firstFrameSent = true;- }+ // if (sendFrameMeta) {+ // writeFrameMeta(fd, bufferInfo, codecBuffer.remaining());+ // }++ // // IO.writeFully(fd, codecBuffer);+ // if ((bufferInfo.flags & MediaCodec.BUFFER_FLAG_CODEC_CONFIG) == 0) {+ // // If this is not a config packet, then it contains a frame+ // firstFrameSent = true;+ // }
}
} finally {
if (outputBufferId >= 0) {
@@ -198,7 +205,7 @@ public class ScreenEncoder implements Device.RotationListener {
headerBuffer.putLong(pts);
headerBuffer.putInt(packetSize);
headerBuffer.flip();
- IO.writeFully(fd, headerBuffer);+ // IO.writeFully(fd, headerBuffer);
}
private static MediaCodecInfo[] listEncoders() {
And here is the output using the same size and bit rate with the default value of my Web client:
When --print-fps is used, Scrcpy won't report them as skipped frames, because it doesn't use VSync, so it doesn't care if the frame is truly displayed. Here I captured the original video stream with scrcpy -r (left) and what Scrcpy shows with OBS (right). the device (a Samsung S9), my PC monitor, and OBS are all set to 60FPS, then the video was slow down to 2FPS. It's pretty clear that while the left video moves every frame, the right video moves every two frames, or sometime one, sometime three.
Environment
Describe the bug
Recently I added a skipped frame counter to my Web implementation of Scrcpy client. Because rendering images on Web is slow, I have VSync on to avoid unnecessary works.
Then I found that the number of skipped frames is insanely high. On my Mi 11 with a 120FPS display and my PC monitor running at 144Hz, usually only about 70 frames are rendered, and the other 50 skipped.
In my further investigation, I found that the time those frames arrive are inconsistent. It's almost like every two frames are sent together. Here is part of my log:
So, I also tested with the official client, the results are same. I modified server code to output time took for each frame to encode, video output is also completely disabled to minimize interference.
And here is the output using the same size and bit rate with the default value of my Web client:
When
--print-fpsis used, Scrcpy won't report them as skipped frames, because it doesn't use VSync, so it doesn't care if the frame is truly displayed. Here I captured the original video stream withscrcpy -r(left) and what Scrcpy shows with OBS (right). the device (a Samsung S9), my PC monitor, and OBS are all set to 60FPS, then the video was slow down to 2FPS. It's pretty clear that while the left video moves every frame, the right video moves every two frames, or sometime one, sometime three.Untitled.mp4