Proxmox ZFS cache reduce to free-up RAM

    As mostly we do not use ZFS on Proxmox, except system volumes, ARC just wastes LOTS (50%) of RAM, also we cannot see real RAM usage by VM on hosts.

    Starting from Proxmox v8.1, new installations of Proxmox apply limit on ZFS cache ARC size to 10% of total RAM amount, also 16GB limit.
    But older installations, even if upgraded to 8.x, are still using 50% of host RAM amount as ZFS cache, also on host RAM usage it is displayed as used.

    You can control it by updating files on hosts (works almost at once, but will be lost after reboot): 
    echo 2000000000 > /sys/module/zfs/parameters/zfs_arc_min
    echo 4000000000 > /sys/module/zfs/parameters/zfs_arc_max
    Official recommendations from Proxmox "We usually recommend a base of 2 GiB plus 1 GiB per TiB of storage."
    Value in parameters are expressed in Bytes, small numbers like 4000000 will be just ignored and default values will be used instead of them.

    Do not set values to zero !!!   Zero is interpreted as default ( 50% of RAM).  Set Max > Min.

    For persistent changes You need to create/modify this file (which will by applied after reboot)
    /etc/modprobe.d/zfs.conf
    options zfs zfs_arc_max=4000000000
    options zfs zfs_arc_min=2000000000

    You can check real ARC cache size by this command on host (search for 'arcsz' value in output):
    arcstat -a

    Real life example:

    Veröffentlicht