GPU MEMORY MONITOR INSTALL
GPU MEMORY MONITOR CODE

GPU MEMORY MONITOR INSTALL

In order to make my life a bit easier, I tried to to look for AMIs that already have NVIDIA drivers installed, so that I don't have to install it myself. There we can find the base AMI, the instance used to create the AMI, the storage configuration, and the scripts used to configure the instance. The file packer/ contains the specification of the AMI. instance, ebs volume, ssh key pair) afterwards. It has capabilities to provision an instance with the base AMI specified, run scripts through ssh, start the process of AMI creation, and clean everything up (e.g.

GPU MEMORY MONITOR CODE

Packer is a great tool to achieve Infrastructure as Code principles on AMI creation step.

The second one has infrastructure as code in terraform to provision an instance using the new created AMI for testing purposes. The first one contains all the necessary files to create the the AMI based on Amazon Linux 2 using a tool called packer. In the project we have two main directories like this:Įnter fullscreen mode Exit fullscreen mode I've created a project (available here) showing how we can create an AMI with CloudWatch agent for RAM and filesystem monitoring, and a custom service called gpumon to gather GPU metrics and send them to AWS CloudWatch. In case of GPU-accelerated applications (like Machine Learning apps), this problem goes even further, since you also don't have any access to GPU metrics, which is critical to guarantee the reliability of the system (e.g., the total GPU memory consumption can lead to the crash of any application running on the GPU). By default, you don't have the monitoring of some basic information, like RAM and filesystem usage (which can be a valuable information to prevent an instance malfunction due to lack of resources). You only have access to CPU utilization, network transfer rates and disk reads/writes.

If you have used provisioned instances on AWS before, you know that the default metrics monitored are kind of limited.

Gpu memory monitor