[GH-ISSUE #103 ] VAAPI and AMD Issues #69

Author

Owner

@salty2011 commented on GitHub (Aug 28, 2024):

hey @ABeltramo

That fair its not an exact 1:1 comparison with the duel gpu setup. Was noting that none of the issues were present using the nvenc encoder so seemed less likely to be a client problem thats all.

Prior the update, I was using the dev-multi-gpu build from several weeks ago which as the time I believe had only the updated gstreamer build that corrected the issue with wide screen resolutions. In the testing there, the performance was notably better.... but the grey screen issue was present.

Let me do the above testing and will report back

@salty2011 commented on GitHub (Aug 28, 2024): hey @ABeltramo That fair its not an exact 1:1 comparison with the duel gpu setup. Was noting that none of the issues were present using the nvenc encoder so seemed less likely to be a client problem thats all. Prior the update, I was using the dev-multi-gpu build from several weeks ago which as the time I believe had only the updated gstreamer build that corrected the issue with wide screen resolutions. In the testing there, the performance was notably better.... but the grey screen issue was present. Let me do the above testing and will report back

bsakaguchi commented

Author

Owner

@salty2011 commented on GitHub (Aug 28, 2024):

Hey @ABeltramo

Apologies for some confusion, I have gone back and repeated testing method better.

Hardware

NVIDIA 1080
AMD 6900XT

All tests were done using H265 and current stable wolf images

Notes

NVIDIA Only: performed at 55FPS for the stream and game play seemed to be allow low. However given the age of the card and its doing Rendering and Encoding, this performance seems in line given the age of the card, vram and it doing both tasks. Also no issue with picture going to grey
AMD Only: performed at 91fps for the video stream, but shortly after game start screen goes to grey. Interestingly if I lower games resolution down to 1920x1080 and set all quality to low the grey issue does not occur. However i can replicate similar issue across multiple games...
NVIDIA Encoding, AMD Rendering: performed at 99FPS for the video stream and no video going to grey. so basically perfect.

Going forward I will use the duel setup as seems to yield the best performance, however was reporting the AMD / VAAPI given others with AMD cards are likely to have the issue as well.

Have attached the stack config and logs for all of the tests
wolf-nvidiaonly-stack.txt
wolf-amdonly_logs.txt
wolf-amdonly-stack.txt
wolf-nvenc_amdrender_logs.txt
wolf-nvenc_amdrender-stack.txt
wolf-nvidiaonly_logs.txt

@salty2011 commented on GitHub (Aug 28, 2024): Hey @ABeltramo Apologies for some confusion, I have gone back and repeated testing method better. **Hardware** - NVIDIA 1080 - AMD 6900XT All tests were done using H265 and current stable wolf images **Notes** - **NVIDIA Only:** performed at 55FPS for the stream and game play seemed to be allow low. However given the age of the card and its doing Rendering and Encoding, this performance seems in line given the age of the card, vram and it doing both tasks. Also no issue with picture going to grey ![wolf-nvidiaonly-cnc](https://github.com/user-attachments/assets/2ea994af-9836-48a1-8740-89fad07889a3) - **AMD Only:** performed at 91fps for the video stream, but shortly after game start screen goes to grey. Interestingly if I lower games resolution down to 1920x1080 and set all quality to low the grey issue does not occur. However i can replicate similar issue across multiple games... ![wolf-amdonly-cnc](https://github.com/user-attachments/assets/afb6d3f7-d646-4fcf-b867-e7fbd5990655) - **NVIDIA Encoding, AMD Rendering:** performed at 99FPS for the video stream and no video going to grey. so basically perfect. ![wolf-nvenc_amdrender-cnc](https://github.com/user-attachments/assets/63c7412d-26c3-4d10-97af-3bb1ebbc00b8) Going forward I will use the duel setup as seems to yield the best performance, however was reporting the AMD / VAAPI given others with AMD cards are likely to have the issue as well. Have attached the stack config and logs for all of the tests [wolf-nvidiaonly-stack.txt](https://github.com/user-attachments/files/16781881/wolf-nvidiaonly-stack.txt) [wolf-amdonly_logs.txt](https://github.com/user-attachments/files/16781883/wolf-amdonly_logs.txt) [wolf-amdonly-stack.txt](https://github.com/user-attachments/files/16781886/wolf-amdonly-stack.txt) [wolf-nvenc_amdrender_logs.txt](https://github.com/user-attachments/files/16781887/wolf-nvenc_amdrender_logs.txt) [wolf-nvenc_amdrender-stack.txt](https://github.com/user-attachments/files/16781888/wolf-nvenc_amdrender-stack.txt) [wolf-nvidiaonly_logs.txt](https://github.com/user-attachments/files/16781889/wolf-nvidiaonly_logs.txt)

bsakaguchi commented

Author

Owner

@ABeltramo commented on GitHub (Aug 28, 2024):

Thanks for testing this out, I'm going to keep this open as a reminder to change the default vaapi pipeline and to give it a go at testing out AMD on AWS (since I don't have an AMD GPU available).
I'm glad you are able to get some good performances using the dual GPU setup!

@ABeltramo commented on GitHub (Aug 28, 2024): Thanks for testing this out, I'm going to keep this open as a reminder to change the default `vaapi` pipeline and to give it a go at testing out AMD on AWS (since I don't have an AMD GPU available). I'm glad you are able to get some good performances using the dual GPU setup!

bsakaguchi commented

Author

Owner

@salty2011 commented on GitHub (Aug 29, 2024):

No problem,

If you need me to do any future testing, happy to do so, I now have 3 wolf stack pre setup to quickly test the above scenario's

@salty2011 commented on GitHub (Aug 29, 2024): No problem, If you need me to do any future testing, happy to do so, I now have 3 wolf stack pre setup to quickly test the above scenario's

bsakaguchi commented

Author

Owner

@StormLaan commented on GitHub (Aug 30, 2024):

Hello,

I've just installed the latest version of Wolf, but I'm encountering a black screen or an issue similar to the one shown in the first image of this thread when starting a game via Steam. My setup includes a 7950X processor and an RX 7800 XT graphics card.

To install Wolf, I used Portainer with the Docker Compose configuration provided in the Quickstart Guide.

In the wolf-wolf-1 container, I see repeated warnings like this:
(0:48:09.260856139 1 0x7f10041f9250 WARN appsrc gstappsrc.c:2469:gst_app_src_push_internal:<wolf_wayland_source> Dropping old item buffer: 0x7f10388e4510, pts 0:46:02.949889482, dts 0:46:02.949889482, dur 0:00:00.016666666, size 8294400, offset none, offset_end none, flags 0x0)

Additionally, here is the log from WolfSteam_10730337829953401807 for further reference.
_WolfSteam_10730337829953401807_logs (1).txt

@StormLaan commented on GitHub (Aug 30, 2024): Hello, I've just installed the latest version of Wolf, but I'm encountering a black screen or an issue similar to the one shown in the first image of this thread when starting a game via Steam. My setup includes a 7950X processor and an RX 7800 XT graphics card. To install Wolf, I used Portainer with the Docker Compose configuration provided in the Quickstart Guide. In the wolf-wolf-1 container, I see repeated warnings like this: (0:48:09.260856139 1 0x7f10041f9250 WARN appsrc gstappsrc.c:2469:gst_app_src_push_internal:<wolf_wayland_source> Dropping old item buffer: 0x7f10388e4510, pts 0:46:02.949889482, dts 0:46:02.949889482, dur 0:00:00.016666666, size 8294400, offset none, offset_end none, flags 0x0) Additionally, here is the log from WolfSteam_10730337829953401807 for further reference. [_WolfSteam_10730337829953401807_logs (1).txt](https://github.com/user-attachments/files/16816450/_WolfSteam_10730337829953401807_logs.1.txt)

bsakaguchi commented

Author

Owner

@salty2011 commented on GitHub (Aug 30, 2024):

hey @Vlekkie2005

Thanks for the information,

Any chance you can attach a screen of the issue and the logs from the wolf container. Like me you are also using portainer so when you bring up the container logs be sure to increase the lines from default of 100 to like 10000.

The reason for the extra information is the logs include information about the gstreamer pipeline settings that were used along with the additional errors. The screen is also for context.

I have found that with different games the issues appears differently. For example game I am testing in the screenshot is CnC Kanes Wrath as it presents the issues the worst. I have found that if I lower the resolution to 1920x1080 and set the graphics quality to low the issue does present at all. In other games like Doom 2016 the issue is most noticeable in load screens

@salty2011 commented on GitHub (Aug 30, 2024): hey @Vlekkie2005 Thanks for the information, Any chance you can attach a screen of the issue and the logs from the wolf container. Like me you are also using portainer so when you bring up the container logs be sure to increase the lines from default of 100 to like 10000. The reason for the extra information is the logs include information about the gstreamer pipeline settings that were used along with the additional errors. The screen is also for context. I have found that with different games the issues appears differently. For example game I am testing in the screenshot is CnC Kanes Wrath as it presents the issues the worst. I have found that if I lower the resolution to 1920x1080 and set the graphics quality to low the issue does present at all. In other games like Doom 2016 the issue is most noticeable in load screens

bsakaguchi commented

Author

Owner

@salty2011 commented on GitHub (Aug 30, 2024):

@ABeltramo > was looking through the issue list for gstreamer vah265 plugin and came across this
https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1991

This could be culprit to the overshoot issue we spoke previously which we theorized may be the cause . Looking at the issue (at least for obs) switching to vaapih265enc did seem to resolve the issue. Concerningly this issue was logged nearly a year ago and doesn't appear any further comment.

I'm going to attempt building the current wolf container to include this plugin and do some testing.

@salty2011 commented on GitHub (Aug 30, 2024): @ABeltramo > was looking through the issue list for gstreamer vah265 plugin and came across this https://gitlab.freedesktop.org/gstreamer/gstreamer/-/issues/1991 This could be culprit to the overshoot issue we spoke previously which we theorized may be the cause . Looking at the issue (at least for obs) switching to vaapih265enc did seem to resolve the issue. Concerningly this issue was logged nearly a year ago and doesn't appear any further comment. I'm going to attempt building the current wolf container to include this plugin and do some testing.

bsakaguchi commented

Author

Owner

@StormLaan commented on GitHub (Aug 30, 2024):

hey @salty2011

Thanks for the reply.

Here are some buggy/not working things and the logs.

Here are my logs(after playing baldurs gate 3 and factorio)
_wolf-wolf-1_logs (8).txt

baldurs gate 3 = black screen (sometimes and other times it does work)

I tested factorio = buggy

and i tested multiple users but i got no video recieved error or games would just not launch (sorry that it is in dutch)

@StormLaan commented on GitHub (Aug 30, 2024): hey @salty2011 Thanks for the reply. Here are some buggy/not working things and the logs. Here are my logs(after playing baldurs gate 3 and factorio) [_wolf-wolf-1_logs (8).txt](https://github.com/user-attachments/files/16821225/_wolf-wolf-1_logs.8.txt) baldurs gate 3 = black screen (sometimes and other times it does work) I tested factorio = buggy ![image](https://github.com/user-attachments/assets/90382b4d-da3b-400d-a5a4-97467de06164) and i tested multiple users but i got no video recieved error or games would just not launch (sorry that it is in dutch) ![image](https://github.com/user-attachments/assets/f4b545bc-193c-43da-9a49-f20e0998abf7)

bsakaguchi commented

Author

Owner

@ABeltramo commented on GitHub (Aug 30, 2024):

Can you confirm just a few things first?

Have you edited the config.toml with the updated pipeline for VAAPI that was posted before?

######################
# Default settings for the main encoders
# To avoid repetition between H264, HEVC and AV1 encoders

[gstreamer.video.defaults.vaapi]
video_params = """
queue !
vapostproc !
video/x-raw(memory:VAMemory), chroma-site={color_range}, width={width},
height={height}, format=NV12, colorimetry={color_space}
\
"""

Have you changed the app_state_folder to be shared between clients in the config.toml?
- Have you manually changed anything else in the config file btw?
Have you tried lowering the resolution, does that help in any way like it does for @salty2011?

@ABeltramo commented on GitHub (Aug 30, 2024): Can you confirm just a few things first? - Have you edited the `config.toml` with the updated pipeline for VAAPI that was posted before? ```toml ###################### # Default settings for the main encoders # To avoid repetition between H264, HEVC and AV1 encoders [gstreamer.video.defaults.vaapi] video_params = """ queue ! vapostproc ! video/x-raw(memory:VAMemory), chroma-site={color_range}, width={width}, height={height}, format=NV12, colorimetry={color_space} \ """ ``` - Have you changed the `app_state_folder` to be shared between clients in the `config.toml`? - Have you manually changed anything else in the config file btw? - Have you tried lowering the resolution, does that help in any way like it does for @salty2011?

bsakaguchi commented

Author

Owner

@StormLaan commented on GitHub (Aug 30, 2024):

I just tried the updated config.toml to the updated pipeline for VAAPI. baldurs gate 3 still has the black screen and factorio still has the same problem

I have not change the app_state_folder or any other config file

in 720p factorio is a bit less buggy and baldurs gate 3 still has a black screen

"The 'No Video Received' issue occurs when I launch Steam, another user launches Steam on the same network, and then I quit and relaunch Steam."

@StormLaan commented on GitHub (Aug 30, 2024): I just tried the updated config.toml to the updated pipeline for VAAPI. baldurs gate 3 still has the black screen and factorio still has the same problem I have not change the app_state_folder or any other config file in 720p factorio is a bit less buggy and baldurs gate 3 still has a black screen "The 'No Video Received' issue occurs when I launch Steam, another user launches Steam on the same network, and then I quit and relaunch Steam."

bsakaguchi commented

Author

Owner

@StormLaan commented on GitHub (Aug 30, 2024):

I have tested some additional games:
helldivers - works
hollow knight - works
hogwarts legacy - works
doom 2016 - works
dragon dogma - works
cyberpunk - black screen

I think the cause of the black screen is the windowed launcher's both cyberpunk and Baldur's gate 3 have a windowed launcher.

Only factorio has the too large packets problem that i noticed.

@StormLaan commented on GitHub (Aug 30, 2024): I have tested some additional games: helldivers - works hollow knight - works hogwarts legacy - works doom 2016 - works dragon dogma - works cyberpunk - black screen I think the cause of the black screen is the windowed launcher's both cyberpunk and Baldur's gate 3 have a windowed launcher. Only factorio has the too large packets problem that i noticed.

bsakaguchi commented

Author

Owner

@ABeltramo commented on GitHub (Aug 30, 2024):

Thanks for the report, the multiuser issue is a silly bug; I've opened another issue to track it (https://github.com/games-on-whales/wolf/issues/109) and I'm going to fix it as soon as I have some free time in the next few days.

As for the AMD specific issues, I'll try it with an AWS instance, I'm in the middle of another PR, though, and I'd like to finish that first. Odd that it's tied to the games, I wonder if it's just because some games compress better than others.

I saw you've got AV1, is that encoder causing the same issues as HEVC? You can force it in the Moonlight client settings even if you don't have an HW decoder on your client machine.

@ABeltramo commented on GitHub (Aug 30, 2024): Thanks for the report, the multiuser issue is a silly bug; I've opened another issue to track it (https://github.com/games-on-whales/wolf/issues/109) and I'm going to fix it as soon as I have some free time in the next few days. As for the AMD specific issues, I'll try it with an AWS instance, I'm in the middle of another PR, though, and I'd like to finish that first. Odd that it's tied to the games, I wonder if it's just because some games compress better than others. I saw you've got AV1, is that encoder causing the same issues as HEVC? You can force it in the Moonlight client settings even if you don't have an HW decoder on your client machine.

bsakaguchi commented

Author

Owner

@ABeltramo commented on GitHub (Aug 31, 2024):

I've pushed a fix for the multi-user issue, you can test it out by checking out the

ghcr.io/games-on-whales/wolf:dev-fix-109

image. I'll test it a bit more and merge it to :stable if nobody reports any issue

@ABeltramo commented on GitHub (Aug 31, 2024): I've pushed a fix for the multi-user issue, you can test it out by checking out the ``` ghcr.io/games-on-whales/wolf:dev-fix-109 ``` image. I'll test it a bit more and merge it to `:stable` if nobody reports any issue

bsakaguchi commented

Author

Owner

@StormLaan commented on GitHub (Aug 31, 2024):

Thanks for the great work.

I did some further testing.

dev-fix-109:
When two devices are connected to wolf and one uses ctrl+alt+shif+q or alt+f4 to quit out of moonlight is causing wolf to crash from what i can see.
_wolf-wolf-1_logs (10).txt

stable build / dev-fix-109:
Forcing AV1 in moonlight gets the same error as no video received.
Games like cyberpunk/Baldur's gate 3 that launches their own launcher to launch their games

@StormLaan commented on GitHub (Aug 31, 2024): Thanks for the great work. I did some further testing. dev-fix-109: When two devices are connected to wolf and one uses ctrl+alt+shif+q or alt+f4 to quit out of moonlight is causing wolf to crash from what i can see. [_wolf-wolf-1_logs (10).txt](https://github.com/user-attachments/files/16824786/_wolf-wolf-1_logs.10.txt) stable build / dev-fix-109: Forcing AV1 in moonlight gets the same error as no video received. Games like cyberpunk/Baldur's gate 3 that launches their own launcher to launch their games

bsakaguchi commented

Author

Owner

@ABeltramo commented on GitHub (Sep 2, 2024):

@Vlekkie2005 you are good at hunting bugs, can I run you on CI? 😅

I've just pushed an additional fix for that exception that you've triggered, could you docker pull again from that branch and let me know if that fixes the multi-user issues that you're experiencing?

@ABeltramo commented on GitHub (Sep 2, 2024): @Vlekkie2005 you are good at hunting bugs, can I run you on CI? 😅 I've just pushed an additional fix for that exception that you've triggered, could you `docker pull` again from that branch and let me know if that fixes the multi-user issues that you're experiencing?

bsakaguchi commented

Author

Owner

@StormLaan commented on GitHub (Sep 2, 2024):

Thanks @ABeltramo it works perfectly as far as i can see.

And i "solved" bg3 from not launching pressing alt+f4 on the black screen with Capture system keyboard shortcuts enabled or using "--skip-launcher" in the starting parameters in steam.

@StormLaan commented on GitHub (Sep 2, 2024): Thanks @ABeltramo it works perfectly as far as i can see. And i "solved" bg3 from not launching pressing alt+f4 on the black screen with Capture system keyboard shortcuts enabled or using "--skip-launcher" in the starting parameters in steam.

bsakaguchi commented

Author

Owner

@ABeltramo commented on GitHub (Sep 2, 2024):

Thanks for checking that out, I'm glad it works!

I'm going to merge that then, and we can keep this open for the AMD specific issues..

@ABeltramo commented on GitHub (Sep 2, 2024): Thanks for checking that out, I'm glad it works! I'm going to merge that then, and we can keep this open for the AMD specific issues..

bsakaguchi commented

Author

Owner

@lyj0309 commented on GitHub (Sep 10, 2024):

Hello,

I've just installed the latest version of Wolf, but I'm encountering a black screen or an issue similar to the one shown in the first image of this thread when starting a game via Steam. My setup includes a 7950X processor and an RX 7800 XT graphics card.

To install Wolf, I used Portainer with the Docker Compose configuration provided in the Quickstart Guide.

In the wolf-wolf-1 container, I see repeated warnings like this: (0:48:09.260856139 1 0x7f10041f9250 WARN appsrc gstappsrc.c:2469:gst_app_src_push_internal:<wolf_wayland_source> Dropping old item buffer: 0x7f10388e4510, pts 0:46:02.949889482, dts 0:46:02.949889482, dur 0:00:00.016666666, size 8294400, offset none, offset_end none, flags 0x0)

Additionally, here is the log from WolfSteam_10730337829953401807 for further reference. _WolfSteam_10730337829953401807_logs (1).txt

hi, i use latest image , and have this issue, i use app_state_folder in second pc have this issue, first is normal

@lyj0309 commented on GitHub (Sep 10, 2024): > Hello, > > I've just installed the latest version of Wolf, but I'm encountering a black screen or an issue similar to the one shown in the first image of this thread when starting a game via Steam. My setup includes a 7950X processor and an RX 7800 XT graphics card. > > To install Wolf, I used Portainer with the Docker Compose configuration provided in the Quickstart Guide. > > In the wolf-wolf-1 container, I see repeated warnings like this: (0:48:09.260856139 1 0x7f10041f9250 WARN appsrc gstappsrc.c:2469:gst_app_src_push_internal:<wolf_wayland_source> Dropping old item buffer: 0x7f10388e4510, pts 0:46:02.949889482, dts 0:46:02.949889482, dur 0:00:00.016666666, size 8294400, offset none, offset_end none, flags 0x0) > > Additionally, here is the log from WolfSteam_10730337829953401807 for further reference. [_WolfSteam_10730337829953401807_logs (1).txt](https://github.com/user-attachments/files/16816450/_WolfSteam_10730337829953401807_logs.1.txt) hi, i use latest image , and have this issue, i use `app_state_folder` in second pc have this issue, first is normal

bsakaguchi commented