Great sleuthing job! I had not seen that page before. I've had problems with users having incompatible display drivers (or older hardware that didn't support the latest drivers) and now I see why:
"or example, an application built against the CUDA 3.2 SDK will continue to function even on today’s driver stack. On the other hand, the CUDA runtime does not provide these guarantees. If your application dynamically links against the CUDA 9.2 runtime, it will only work in the presence of the dynamic 9.2 CUDA runtime on the target system. If the runtime was statically linked into the application, it will function on a minimum supported driver, and any driver beyond"
Looks like static linking would be best for compatibility. Also for compatibility I've noticed that I need to specify the all the GPUs that the application will be compatible with using the arch and code compiler flags like this:
--generate-code arch=compute_53,code=sm_53
https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#options-for-steering-gpu-code-gene...