Topic: Why is there no external solution for shader compilation stutter?

I've been wondering for a while why there isn't an external way to compile shaders for a game—either for individual games, as a mod, or as a tool for specific graphics engines. I'm not aware of a single example where this is possible.

Why is that? We see complete graphics and gameplay mods, UEVR mods, and similar things. But no options for shader compilation.

I don't have much knowledge about the technology, so I'm very interested in understanding why this has not been possible so far.

@alex-battaglia often mentions that even with hardware power, nothing can be done about it. It would be really interesting to understand why we seem so helpless when it comes to this issue.

There is Alex's Shader Butler concept, which Valve apply a version of for the Steam Deck and Intel plan to roll out something similar for their GPUs later this year and consoles ship with fixed shaders so never need to compile on the fly.

There is essentially no technical reason why Epic could not make it easier for devs to assemble the list of shaders to compile on startup/after a driver update, or devs could be more diligent in doing a shader pass, or Nvidia or AMD to fix this at the driver level for games. The reason it has not been done at scale yet is because there has not been a big enough financial incentive for anyone to do it.

My understanding is that unlike consoles, which have fixed graphics hardware and therefore can have games that include the shaders as part of the initial install and updates, PC has many different types of graphics cards, not only from Intel, NVIDIA and AMD but also from different generations that all require the shaders to be uniquely generated when the game loads up for the first time (assuming the game even has a pre-shader compilation step, some older games don't even get that right), when you update your graphics drivers and sometimes even when the game itself is patched/updated.

It's an unfortunate weakness of the PC hardware in my experience when I can load up a PS5 games such as The Last of Us Part 1 and be playing it within seconds but on PC I have to wait 20+ minutes for the shaders to compile because I had updated my graphics drivers since the last time I played it (incidentally, these long shader compilation times of 10-20 minutes in games like the Uncharted Legacy of Thieves Collection and The Last of Us Part 1 are the reason I stopped playing them... it just takes too damn long on my Intel i5-13600KF).

There is supposed to be a solution coming, likely as a result of the next Xbox being a console/PC hybrid which means that shader compilation needs to be better handled. I would much rather download pre-compiled shaders from NVIDIA for my RTX 4080 than have to wait 1-20 minutes, depending on the game, before I can play it because that would obviously be much quicker. These shaders caches are often little more than a few hundred megabytes in size which would take no time to download on modern fibre internet connections.

Is it even possible for NVIDIA, Intel and AMD to offer downloadable shader packs though with each new driver? I feel that if it is then that would be not only a better and faster solution but presumably less wasteful on energy since one set of shaders can serve tens of thousands of users instead of them having to use their hardware to generate them individually.

Yes, I know the differences between consoles and PC. That is not the problem.

I have no problem waiting. I would even wait for 2 hours if it meant that I no longer have stutter in the game.

That's also what my question in the thread title is aimed at. Many games either have no shader compilation at all or are incomplete to speed things up. Both solutions result in stutter—sometimes more, sometimes less.

I would like to have absolutely no stutter. For that, I would be willing to wait a long time if necessary.

The fundamental question is why it is not possible to manually initiate this, perhaps through an external mod or tool. I am not aware of any cases where this is really possible. And that's the point I don't understand. Is there no interest in it? Is it technically not feasible?

I've not looked into it but if UE games store a map file of some sort that contains key: value data e.g. { id: file } for every used shader, in theory it should be possible to create an external program (Rust would be a good language candidate) that traverses this map, and compiles every shader using an API call that I imagine must exist (I looked it up...its in dxcompiler.dll...pretty self explanatory)

If you're interested...Get hold of Claude Code and ask it to write you such program.

@theincredibleaf
I tried that, it does not understand enough to be able to do it.