-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
🐛 useSkiaFrameProcessor renders black screen using front cameras #2951
Comments
Same issue. |
I've updated my version to "4.2.0" also. By chance, I tipped my phone from portrait to landscape whilst using the front camera and it started working. As soon as I tip it back to portrait it goes black. Try that out @DarkShtir. I assume therefore it's something to do with the new orientation changes in the recent release. Will have an investigate of the latest changes |
@khlling this works, but the same time is not easy and comfortable to use. |
@DarkShtir Agreed I'm sure it's not the intended behaviour. What changes did you have to make in 4.0.5 to get it to work? Can you share your patch files? |
No problem, is local fix for project, to use skiaFrameProcessor |
I had the same blank screen issue on Android 14 (Samsung A53) and resolved the issue by changing minSdkVersion from 24 to 26 in android/build.gradle. |
Also add the following would help: |
@tsnguyenducphuong wait I thought VisionCamera/Skia will throw an error if |
Either way, the issue from OP (@khlling) is not related to minSdkversion as it renders fine on back camera, but not on front. I think this has to do with the orientation/matrix code - @khlling can you play around with this function to figure out where it goes wrong? react-native-vision-camera/package/src/skia/useSkiaFrameProcessor.ts Lines 61 to 82 in 90be06b
I unfortunately don't have the free time to look into this right now, but if you find a solution in there I'd appreciate a PR! Thanks |
Maybe it'd also be helpful if you posted |
I can confirm that VisionCamera/Skia does throw an error for this. I had to update the Post this update I'm getting a blank front camera preview. |
No worries @mrousavy let me do a bit of digging. Thanks for giving me a hint of where to start looking! Will update this issue with logs and investigation. |
In my case, I had first the blank screen issue then the black screen and solved with the following steps: (I am using Vision Camera version 4.2.1)
Solve black screen:
|
@mrousavy it's interesting because in the video use-case the error is clean and concise, I assume it would be the same for the vision-camera integration? Could it be because of an older version of Skia (I did improve this recently I think) |
True, maybe an old RN Skia version! I'll do more research thanks |
I'm experiencing the same bug, but on iOS. When a Skia frame processor is connected, the camera stays black as long as the iPhone is in portrait. Some observations: When I rotate the iPhone to landscape (which our UI doesn't support), the camera appears, but is rotated by 90 degrees. On iPad, where we support both portrait and landscape mode for the UI, everything works as expected, though. When I set I'm not sure if that's expected behavior for apps that don't support landscape mode, but I've also noticed From within the frame processor, logging All in all, there seems to be something going wrong with detecting the device's orientation, leading to wrong transforms on the Skia frame. I just rebuilt the app with the newest react-native-skia release (1.3.4) which didn't fix it, unfortunately. Edit: My iPhone runs on iOS 17 while my iPad still runs on iOS 16. So, if this is related to #2972, this might be why it works fine on the iPad. |
After I digged a little into the different orientation values provided by the library, I realized most of my previous comment is actually the expected behavior and the problem must indeed lie in the frame rotation in useSkiaFrameProcessor.ts. I played around a little and came up with this: switch (frame.orientation) {
case 'portrait':
case 'portrait-upside-down':
canvas.scale(frame.height / frame.width, frame.height / frame.width)
canvas.rotate(90,0,0)
canvas.translate(0, -(frame.height * 2))
break
case 'landscape-left':
case 'landscape-right':
canvas.translate(frame.height, 0)
canvas.rotate(90, 0, 0)
break
default:
throw new Error(`Invalid frame.orientation: ${frame.orientation}!`)
} This seems to work great, however, two caveats:
So, this isn't even close to a full fix or PR ready yet, just some observations. |
Hello, I have the same problem, and you can solve it by changing the logic of the landscape-left case:
but I run into the problem that the camera creates a mirror effect, and I can't solve it.
but the image cannot be reversed |
You want to disable mirroring, but pass |
I'm sorry @mrousavy. I tried isMirrored={false} and isMirrored={true} in the camera component, I haven't found out if it can be changed in the preview |
Ah, so the Preview is not mirrored with a Skia Frame Processor, but is properly mirrored without a Skia FP - is that what you're saying? |
is there a workaround for this bug ? |
Seeing this in iOS, running in an expo development build
I'm in early days of development, so it's not a show-stopper at the moment, let me know if I can provide any useful info. |
you can play around with this function to figure out where it goes wrong: react-native-vision-camera/package/src/skia/useSkiaFrameProcessor.ts Lines 61 to 82 in 90be06b
|
I was facing this issue on iPad, the screen was cutting off in both landscape left and right, and in upside down the whole camera was giving black screen, I made this adjustment and it worked perfectly in all cases (no need to change rotation configuration) switch (orientation) {
case 'portrait':
canvas.translate( 0,frame.height / 2)
break
case 'landscape-left':
// rotate two flips on (0,0) origin and move X + Y into view again
canvas.translate(0, frame.width)
canvas.rotate(270, 0, 0)
break
case 'portrait-upside-down':
// rotate three flips on (0,0) origin and move Y into view again
canvas.translate(frame.width, frame.height * 1.5)
canvas.rotate(180, 0, 0)
break
case 'landscape-right':
// rotate one flip on (0,0) origin and move X into view again
canvas.translate(frame.height, 0)
canvas.rotate(90, 0, 0)
break
default:
throw new Error(`Invalid frame.orientation: ${frame.orientation}!`)
} |
Face the same issue, found the cause that front camera actaully do a lateral inversion because its need to show picture just like a mirror . You can verify this by drawing something on canvas and changing camera from back to front |
thanks @raslankiwan , I had also this issue on Android phone, in portrait view I was getting black screen. In landscape it was ok. I added your changes in node_modules\react-native-vision-camera\src\skia\useSkiaFrameProcessor.ts and now in portrait I also get camera view |
What's happening?
When calling frame.render() within useSkiaFrameProcessor on an Android phone, the captured frames render as expected when the camera position is set to 'back'. However, setting the camera position to 'front' results in a black screen.
Omitting useSkiaFrameProcessor (i.e., not passing a frame processor as a prop to ReanimatedCamera) renders as expected
Recording and playing back the video output from the front camera works fine, indicating the frames are captured correctly but not rendered properly in real-time.
Reproduceable Code
Relevant log output
Camera Device
Device
Pixel 6 (Android 14)
VisionCamera Version
4.1.0
Can you reproduce this issue in the VisionCamera Example app?
Yes, I can reproduce the same issue in the Example app here
Additional information
The text was updated successfully, but these errors were encountered: