OK, after several hours' toiling, I think I can better describe the problem now and I think the problem does have to do with my installation, I probably left some key module behind:
On my Virtualmin I can choose to enable php modules(5.6,7.0,7.2), I've found out that if I enable 7.0 instead of 7.2, the problem will be no more. Whereas the PHP execution mode is FastCGI and its PHP version is 7.2, also fpa shows PHP version is 7.2.
How apache's php module is working together with FastCGI is still in over my head. I've thought with PHP exe mode set to FastCGI, I could even disable apache's php module, but it turns out in my case I need to enable it and I need to enable it as PHP 7.0. It sounds very weird but it definitely works for me.
But I am not satisfied with merely get it working. I want it to work with 7.2 module. To achieve this, I guess that I probably need to install some additional apache module for php7.2 to get the FastCGI work with apache's PHP7.2 module, but I can't find where to start with. Can anyone give any hint?