With your machine on and hose plus mask in place, just press and hold the raised knob on the machine for about 3 or 4 seconds, and the machine will enter a mask test mode in which it builds up pressure in your mask while displaying either the green smiley face or the red sad face in response to the current leak rate. You can even loosen and or tighten the headgear and see the leak results during this test.
Open mouth has been a problem for many of us. I find it hard to understand your trying a full face mask not solving the open mouth problem. With the high leakage you reported using a FFM, makes me wonder if you may have had the headgear too tight which collapses the seal causing leakage. My top pressure is now set at 19, and mask leakage wasn't a problem for me using the Quattro FFM. In my case, damage to my nose bridge did become the issue. My solution was switching to the Respironics FitLife face mask which solved both leakage and nose damage problems.