Sounds like the regulator is creeping pressure (leaking by the internals). Bummer but no surprises there.
Maybe you should proof your regulator's condition.
A very good rule of thumb on QJs is to use 5.5 to 6.5psi and that will fit with all N&S sizes.
Regards,
HB2
Dissident