Lesson 02: Optimizing RAM Usage on Jetson Orin Nano 8GB

Running large language models (LLMs) on a Jetson Orin Nano with 8GB of RAM requires optimizing available memory. Here are some steps:

Disabling the Desktop GUI

If accessing your Jetson remotely via SSH, you can disable the desktop GUI to free up about 800MB.

• Temporary disable the desktop, run commands in the console, and then re-start the desktop when desired:

  $ sudo init 3 # stop the desktop
  # log your user back into the console (Ctrl+Alt+F1, F2, ect)
  $ sudo init 5 # restart the desktop

• To make this change persistent across reboots:
  

  
  sudo systemctl set-default multi-user.target    # To disable desktop on boot
  sudo systemctl set-default graphical.target     # To enable  desktop on boot
  

Disabling Miscellaneous Services

Disable unused services to free up memory:

sudo systemctl disable nvargus-daemon.service

Mounting Swap

For large models, creating a swap file is beneficial, preferably on NVMe SSD:

• Run these commands to disable ZRAM and create a swap file:
  

  sudo systemctl disable nvzramconfig
  sudo fallocate -l 16G /ssd/16GB.swap
  sudo mkswap /ssd/16GB.swap
  sudo swapon /ssd/16GB.swap
  

• Add the swap file to /etc/fstab for persistence:
  

  /ssd/16GB.swap none swap sw 0 0

These steps will help optimize RAM usage on your Jetson Orin Nano 8GB System.