fix: Inference engine Nitro with Windows with/ without CUDA

[hiro-v]

Windows has serious problem with only CPU and also it cannot use NVIDIA GPU properly

See comments for the test result

[hiro-v]

• CPU:
  
• Jan app usage:
  

Machine:

• OS: Windows 11 Home

• No AMD/ NVIDIA GPU

• CPU: Intel 8th i7

• Memory: 16GB

• Model: TinyLlama, but other model yields the same performance

• Performance: 15 tokens/s (very reasonable)

• It's not laggy as it uses Physical cores (not logical cores which normally on windows they supports hyper-threading which is 2 per CPU, and on Intel 12th+ it has performance/ efficiency cores which is not easy to calculate). This setup helps a lot for normal computer as compute intensive tasks (LLM model) are scheduled to run on performance cores, while other IO intensive tasks (even OS background tasks) on others
  The CPU does not likely to hit 100% and hold at that level (better from what we experienced with current stable release on windows)

However there are some Windows specific problem that we observe:
After windows sleep and wakeup, the perf decrease by apprx. 20%
Windows without power plugged in has perf decrease but I did not have chance to test.

[hiro-v]

• CPU:
  
• NVIDIA GPU VRAM usage with Nitro log
  
• GPU utilization and VRAM consumption
  
• Jan App performance
  

Machine:

• OS: Windows 11 Home
• NVIDIA GPU 3090 with 24GB VRAM
• CPU: Intel 13th i9
• Memory: 64GB

Test:

• Model: Tinyllama
• Nitro version: 0.1.26
• Performance (impressive): 57token/s (almost fully offloaded to NVIDIA GPU)

[hiro-v]

Test result on the laptop without NVIDIA, now I plug the NVIDIA GTX 1050ti 4gb RAM.
The best thing is that I close Jan app and open again, it can use NVIDIA GPU right away without any further installation, which is good UX.

Here is the test result:

• Jan app result (22 tokens/s)
  

• CPU usage
  

• GPU usage
  

• Nitro NVIDIA utilization
  

• Model: TinyLlama

• Perf: 22 tokens/ s