We just saw that there was a HUGE memory leak due to Unity 6000.2.6f2 that we did not see with or 64Go RAM PCs !
We fixed it by downgrading to Unity 6000.0.x (LTS version).
It now works !

Here the link to the closest ressource I found to confirm that it was a Unity side bug : Link
Hope it will help some of you too !

Our game : Link