Greetings programs!

I came back from vacation yesterday and for some reasons a few things have changed on out router. I doubt that anybody has fiddled with the configuration, because only the boss and myself have the password to do any fiddling, apart from that, there is noone in the office who knows anything about the router to attempt that.

There were some connection issues, probably caused by DSL-problems and the people at the office thought it was a good idea to try power-cycles.

None of that however explains a few things I have noticed...

#1: The ntpd no longer starts.
When I set up the router, I configured it to start the busybox ntpd (as client and as server). It did what we needed so I saw no reason to install another package. I tested it several times because I built and installed a few new workstations which are configured to get the time from the router. The configuration is pretty simple (/etc/config/system):

config system
        option  hostname        'heimdall'
        option  timezone        'CET-1CEST,M3.5.0,M10.5.0/3'
        option  log_file        '/mnt/storage/log/syslog'

config timeserver ntp
        list    server          'ptbtime3.ptb.de'
        list    server          'de.pool.ntp.org'
        option  enable_server   '1'
        option  enabled         '1'

As you can see, no big deal. However, the router does not set its own time, nor does it allow other machines to connect and get the router's time. After a reboot, the system time is completely off:

root@heimdall:~# date
Fri Apr  7 13:32:14 CEST 2017

The time servers in the list are however reachable:

root@heimdall:~# ntpd -dqn -p ptbtime3.ptb.de
ntpd: resolved peer ptbtime3.ptb.de to 192.53.103.103
ntpd: sending query to 192.53.103.103
ntpd: reply from 192.53.103.103: offset:+0.002082 delay:0.012552 status:0x24 strat:1 refid:0x50544200 rootdelay:0.000000 reach:0x01
root@heimdall:~# ntpd -dqn -p de.pool.ntp.org
ntpd: resolved peer de.pool.ntp.org to 78.46.53.8
ntpd: sending query to 78.46.53.8
ntpd: reply from 78.46.53.8: offset:+0.005597 delay:0.016232 status:0x24 strat:2 refid:0x83bc03de rootdelay:0.008652 reach:0x01

At the same time, after rebooting (see below), there are no error messages in the log file (router contains a USB-stick for storing logs). Because of this I have absolutely no idea, why the ntpd isn't running. Note that there is plenty of other stuff in the log files apart from ntp though.

Supplementary:
It seems as if wan1 and wan2 take a rather long time to go up. So it is possible that the ntpd is started before there is an internet connection. IMHO however, I should see an error message if this is the reason why the ntpd isn't started.

#2: reboot only works in combination with -f
While this is more a side note (or a further symptom), I thought I'd mention it too, because it is a little ball-busting. A simple "reboot" does nothing. The command is take and seemingly ignored (connection via ssh). Same goes for Lucy. The command is acknowledged but nothing happens. Only when I type "reboot -f" on the console, the router actually reboots. This was not the case earlier either.

Can anyone make sense of this?

Best regards,
Cassi

(Last edited by Proudmouse on 20 Jun 2017, 11:43)

Do you have a script file in your /etc/rc.d/ folder that has the name ntp on it? On my system (OpenWRT 15.05.1), this file is called S98sysntpd.

This script file contains the logic to first read the configuration from the /etc/config/system file, then formulates the command-line arguments of the ntpd program based on these settings, and finally starts the ntpd program with these arguments using a procd helper.

You need to understand that the /etc/config/system file contains the configuration of the ntpd program. It does not contain the actual commands to start it. The actual start-up is done by a specific script file that reads the configuration and starts the program, and is often accompanied by a support script that calls the previous one when specific events occur, such as a network interface comes online.

Check your /etc/rc.d/ folder. Do you have a script file there that starts ntpd? Check LuCI "System -> "Startup", and see if this script file is enabled (i.e. starts automatically at boot)? If you restart your router and then manually start the script from LuCI or from command-line (using e.g. '/etc/init.d/sysntpd start'), then does the time get set correctly?

I have no idea why the 'reboot -f' is required. Perhaps there's a program running that doesn't shut down correctly on a normal reboot? Check the output of 'ps' to see what programs are running, and work forwards from there.

Just to clear this up from the start...
Version (from LuCI): OpenWrt Chaos Calmer 15.05 / LuCI (git-15.248.30277-3836b45)
I tried to update a while back, but it broke the mwan installation and since I need that, I went back to the older version. The documentation is a little thin when it comes to these rather exotic packages. But more to the subject at hand...

Yes, the file exists and it also seems to work fine - just not on startup. I was actually aware that the files in /etc/config/ are not the actual config files (because I have dealt with these before on other distributions and on FreeBSD). I was looking for the script that interprets the generic config file, but wasn't sure where to look. Thanks for lighting the way!

I did both. I can start the ntpd manually via LuCI or via the start script on the console (as in your quote). The time is set, after that I can find the ntpd:

root@heimdall:~# ps w | grep ntp
 6267 root      1464 S    /usr/sbin/ntpd -n -l -S /usr/sbin/ntpd-hotplug -p ptbtime3.ptb.de -p de.pool.ntp.org
 6325 root      1456 S    grep ntp

I have also tested the ntp-server which works after the start too. So basically it boils down to the ntpd not starting at bootup.

Any ideas why this might be - and where the error messages could be hiding?

Yeah, I tried that, but it didn't get me anywhere. The list ps spits out is just as full as it is before I give it the reboot command. Nothing seems to be happening at all...

Cheers!
Cassi

On my router, the ntpd startup scripts contains a few echo statements which apparently echo directly to console output. Since there's no console (other than the serial console) attached to the router when it is booting up, then these messages are probably lost.

Thus, you could check the ntpd startup script and perhaps use the 'logger' command at specific points to write info messages to the system log? You can then use 'logread' to read these messages out. This would at least give you an idea of the execution path of the script during start-up.

There are a few echo statements which indicate invalid configuration, perhaps replacing these with 'logger' commands might help you in trapping if these branches get executed? Or perhaps adding a 'logger' command to the end to signal that the script has completed?

Use 'logger --help' to get the syntax of the command. You can call it directly from the script file.

If the logs show that the script is executed properly, but ntpd just refuses to start, then I'm out of ideas

What if you create a hotplug script to start after the wan is up?
Add a script /etc/hotplug.d/40-ntpd with the following and make it executable

#!/bin/sh

if [ "$INTERFACE" = "wan" ] && [ "$ACTION" = "ifup" ]
then
        /etc/init.d/ntpd restart
fi

Will that actually still work? According to
https://wiki.openwrt.org/doc/techref/hotplug
the hotplug2 daemon was removed something like 4 years ago and replaced by procd. If this just means that the daemon was exchanged, but the functionality remains, then forget I said anything.

Regards,
Cassi