Log File Rotation Weekly Problem (nginx.service start request repeated too quickly, refusing to start)

Hi. Once every week (mostly on weekends) nginx suddenly stops. Mostly ealry in the morning and weekend. If I start it manually everything is working fine. Finally i reproduced this error by Force Log File rotation for all hosted domains on my VPS and Nginx failed (nginx.service start request repeated too quickly, refusing to start.) Why Webmin is executing "service nginx restart" for every Virtual Hosts? How to fix this? This is a bug?

Status: 
Needs review

Comments

Howdy -- how many domains do you have on your server there?

I have 9 Domains on my VPS.

Try disabling logrotate for all your domains, and then re-enabling it again - this should setup a single logrotate config entry that matches all logs, and does only a single restart. This can be done from the command line, with the commands :

virtualmin disable-feature --logrotate --all-domains

virtualmin enable-feature --logrotate --all-domains

I executed the commands via terminal. but as you can see from this image --> http://postimg.org/image/65ciuk33t/ Virtualmin is still creating logrotate for each domains separately and adding "service nginx restart" as post command for each domain. for a temporary fix i removed all post commands and keep "service nginx restart" for last domain. and try a force logrotate for all domains and it is now working fine.

I am using Debian Linux 8 & Webmin 1.791

Can you check if at System Settings -> Server Templates -> Default Settings -> Log file rotation, that "Share logrotate configuration with all domains?" is set to "Yes"

Share logrotate configuration with all domains is NO! So that was my mistake right? :(

There is any article which describes all important setting to go through after a fresh install?

Thanks for the help & this awesome control panel.

Actually, that should be the default - I will fix that in the next Virtualmin release.

so, if I create many domaIns with subdomains and run virtualmin with nginx, I have to execute after that? I haven't done this yet, and the issue still happens on the latest virtualmin version.

virtualmin disable-feature --logrotate --all-domains

virtualmin enable-feature --logrotate --all-domains

I agree with @hkristjan maybe Nginx restart is not needed if signals can be used?

I have a filling that is the reason for my processes that are multiplying

any news about this?

is safe to switch from "service restart" to "[ -f /var/run/nginx.pid ] && kill -USR1 cat /var/run/nginx.pid"?

I got the same problem too, Nginx didn't restart every night because of log rotation. I have some domains that share the same configuration and others (newer) that was not sharing it.

With the two commands by Jamie I solved... but I can't understand why newer domains were created separated from older... maybe because I changed some options in the default template?

p.s. I'm on Virtualmin 5.99

Yes, you could switch to that command, assuming that a USR1 signal causes Nginx to reload it's config.

It sounds like we should make this the default in Virtualmin actually.

I'm still having issues with this. Virtualmin has first created /etc/logrotate.d/initialdomain.conf and later added new logs to this file also for new domains. But after some time, for some reason, now it creates new logrotate config file every time I add new virtual server. This in time results in error of too many restarts and nginx doesn't start, so I have to start it manually on Sunday morning.

I have later manually renamed /etc/logrotate.d/initialdomain.conf to /etc/logrotate.d/virtualmin and merged all log files to this one to have only one config and consequently one nginx restart, but when I add new virtual server in creates new logrotate config file again.

How can I configure Virtualmin so it will pick up existing logrotatce config and just add new entries to it?

"Share logrotate configuration with all domains?" is set to Yes. Also I have run above 2 commands manually and it did not solve the problem.

In the next Virtualmin release, it will be less picky about re-using existing config logrotate files in this kind of situation.

Any news about this issue? For me it is still creating new configuration file for every newly created virtual server. So I have to merge configuration files manually and if I forget I end up with offline server every Sunday morning (nginx not running).

Also, if I look into /etc/logrotate.d/nginx on Ubuntu it doesn't restart the server, but does:

postrotate invoke-rc.d nginx rotate >/dev/null 2>&1 endscript

Is there any reason why Virtualmin is not doing the same?

Jamie shared in Comment #16 above that the next Virtualmin version would attempt to reuse the existing logrotation config file where possible. Does that help with what you're after?

Sure, I was wondering if this issue was already addressed, since there were many new minor versions released since that comment and the issue still exists.

Ah, I read the timestamp too quickly... I saw his reply on 6/25, and thought it was 9/25, which I thought was one day ago :-)

Unfortunately though, there hasn't been a new Virtualmin release since then. Virtualmin 6.03 is the most recent, which was released a few months ago.

We do hope to release a new version here shortly, I'll ask Joe about how that's coming along!

Yeah, we can and should do a new Virtualmin release ASAP.

While we're on it, how is VIrtualmin / Webmin going to know which config file is the one new lines (path to new log files for newly created virtual server) should be merged into?

For example, I had bit of a mess, some config files (inside /etc/logrotate.d/) were named after firstly created domain (example1.com) which had entries for several domains and then new ones were added to separate config files (example10.com, example11.com, etc) - so I merged all of them to just 'virtualmin'.

My suggestion is, Virtualmin should use fixed filename in '/etc/logrotate.d/filename', like in my case. This is in my opinion better than naming it after firstly added virtual server name, which can be later deleted and causing confusion.

Also, I'm no expert in this, but seems like there is no actual need to restart nginx after weekly logrotate. Instead of putting service nginx restart to postrotate script you could put service nginx rotate, which is more safe.

Using the same file is a good suggestion - I'll do this in the next Virtualmin release.

Using "service nginx rotate" sounds like a good idea, but it looks like that isn't widely supported on all Linux distros.

Hello, I am having this issue repeatedly now as well. I tried the posted steps and then took a look at my log rotate and it looks like this. https://i.imgur.com/hOeK99m.png

Then I just see things like this several times in my logs. https://i.imgur.com/g3VVDAu.png

Anything else I can do to fix this? For the moment I am going to try and just place them all under the same one, but I don't think that is going to be a good permanent fix.

Thanks, -MH

Combining all the domains' logs into a single logroate config is the best way to go.

I don't want to put all logs here but I have similar problem with Nginx, Debian 9.5 Webmin version 1.894 Virtualmin version 6.04 my logs https://www.virtualmin.com/node/59126#comment-804316

System Settings -> Server Templates -> Default Settings -> Log file rotation, is Yes in my config

virtualmin disable-feature --logrotate --all-domains
virtualmin enable-feature --logrotate --all-domains

I can confirm that it solved a problem.

I'm combining same settings like @adamus007p + cron periodicaly for same commands

Feels stupid and redundant as hell