Loader loads ICDs and reports physical devices twice on Linux when sorting is active

[aqnuep]

Describe the bug
There is a long standing bug when Linux sorting takes effect in the loader, particularly when the same ICD JSON is found by the loader through multiple mechanisms.

We noticed this while downstreaming the latest Vulkan Loader code to the Vulkan SC Loader, revealed by this minor change in the test cases introduced by this commit: 4c138ad

The EnvVarICDOverrideSetup.TestOnlyDriverEnvVarInFolder test case started to fail because it reported some of the devices of some ICDs twice.

This test case works (after the modification) by adding one ICD configured through VK_DRIVER_FILES by directory path, and adding two other ICDs with two devices each through VK_DRIVER_FILES but by file name. This test case passes in the Vulkan Loader but now fails in the Vulkan SC Loader, but the bug is actually a general one, and one that exists for a long time.

The reason why the failure does not show up in the Vulkan Loader tests is because the test case runs with Vulkan version 1.0, which does not have GPDP2, therefore the Linux sort logic does not take effect and things happen to work correctly. However, as in Vulkan SC GPDP2 is core even in version 1.0, we see the misbehavior.

There seem to be quite a few questionable things there in the related code, for example, the loader does perform duplicate elimination, but only at the level of the search paths, which in the case of these tests will not show redundancy, as the first ICD is added by directory path, while the others by file path. The latter therefore end up being added twice, once because they are found in the directory, once because they are found by file path.

Therefore in order to weed out such bugs, I think it may be worth considering testing all these cases both with Vulkan version 1.0 and 1.1, as a lot of loader logic depends on the presence of GPDP2.

Environment (please complete the following information):

• OS: Linux
• Bitdepth: 64-bit
• GPU: Any
• Graphics Driver: Any
• SDK or header version if building from repo: main
• Enabled layers: N/A

To Reproduce
Steps to reproduce the behavior:

1. Modify the test case EnvVarICDOverrideSetup.TestOnlyDriverEnvVarInFolder to use Vulkan 1.1, e.g. by calling inst[12].create_info.set_api_version(VK_API_VERSION_1_1); on the used instance wrappers
2. Run the test case and observe that it fails due to some devices being reported twice due to their ICDs being found and loaded twice

VK_LOADER_DEBUG output

...
[Vulkan Loader] INFO | DRIVER:  linux_read_sorted_physical_devices:
[Vulkan Loader] INFO | DRIVER:       Original order:
[Vulkan Loader] INFO | DRIVER:             [0] pd10
[Vulkan Loader] INFO | DRIVER:             [1] pd11
[Vulkan Loader] INFO | DRIVER:             [2] pd00
[Vulkan Loader] INFO | DRIVER:             [3] pd01
[Vulkan Loader] INFO | DRIVER:             [4] pd10
[Vulkan Loader] INFO | DRIVER:             [5] pd11
[Vulkan Loader] INFO | DRIVER:             [6] pd00
[Vulkan Loader] INFO | DRIVER:             [7] pd01
[Vulkan Loader] INFO | DRIVER:             [8] pd0
[Vulkan Loader] INFO | DRIVER:       Sorted order:
[Vulkan Loader] INFO | DRIVER:             [0] pd10  
[Vulkan Loader] INFO | DRIVER:             [1] pd11  
[Vulkan Loader] INFO | DRIVER:             [2] pd00  
[Vulkan Loader] INFO | DRIVER:             [3] pd01  
[Vulkan Loader] INFO | DRIVER:             [4] pd10  
[Vulkan Loader] INFO | DRIVER:             [5] pd11  
[Vulkan Loader] INFO | DRIVER:             [6] pd00  
[Vulkan Loader] INFO | DRIVER:             [7] pd01  
[Vulkan Loader] INFO | DRIVER:             [8] pd0  
...