Presenting local AI-powered software options for tasks such as image & text generation, automatic speech recognition, and frame interpolation.
Benchmarking with TensorRT-LLM
Evaluating the speed of GeForce RTX 40-Series GPUs using NVIDIA’s TensorRT-LLM tool for benchmarking GPU inference performance.
Experiences with Multi-GPU Stable Diffusion Training
Results and thoughts with regard to testing a variety of Stable Diffusion training methods using multiple GPUs.
Can You Run A State-Of-The-Art LLM On-Prem For A Reasonable Cost?
In this post address the question that’s been on everyone’s mind; Can you run a state-of-the-art Large Language Model on-prem? With *your* data and *your* hardware? At a reasonable cost?
Self Contained Executable Containers Using Enroot Bundles
NVIDIA Enroot has a unique feature that will let you easily create an executable, self-contained, single-file package with a container image AND the runtime to start it up! This allows creation of a container package that will run itself on a system with or without Enroot installed on it! “Enroot Bundles”.
NVIDIA 3080Ti Compute Performance ML/AI HPC
For computing tasks like Machine Learning and some Scientific computing the RTX3080TI is an alternative to the RTX3090 when the 12GB of GDDR6X is sufficient. (Compared to the 24GB available of the RTX3090). 12GB is in line with former NVIDIA GPUs that were “work horses” for ML/AI like the wonderful 2080Ti.
Run “Docker” Containers with NVIDIA Enroot
Enroot is a simple and modern way to run “docker” or OCI containers. It provides an unprivileged user “sandbox” that integrates easily with a “normal” end user workflow. I like it for running development environments and especially for running NVIDIA NGC containers. In this post I’ll go through steps for installing enroot and some simple usage examples including running NVIDIA NGC containers.
Quad RTX3090 GPU Power Limiting with Systemd and Nvidia-smi
This is a follow up post to “Quad RTX3090 GPU Wattage Limited “MaxQ” TensorFlow Performance”. This post will show you a way to have GPU power limits set automatically at boot by using a simple script and a systemd service Unit file.
Quad RTX3090 GPU Wattage Limited “MaxQ” TensorFlow Performance
Can you run 4 RTX3090’s in a system under heavy compute load? Yes, by using nvidia-smi I was able to reduce the power limit on 4 GPUs from 350W to 280W and achieve over 95% of maximum performance. The total power load “at the wall” was reasonable for a single power supply and a modest US residential 110V, 15A power line.
RTX3070 (and RTX3090 refresh) TensorFlow and NAMD Performance on Linux (Preliminary)
The GeForce RTX3070 has been released.
The RTX3070 is loaded with 8GB of memory making it less suited for compute task than the 3080 and 3090 GPUs. we have some preliminary results for TensorFlow, NAMD and HPCG.
