How to use NTS
Note: NTS is a new protocol which is still in the draft stage. It's close to being finalised, but details may still change.
There are multiple NTS client implementations out there. Probably the first complete NTS implementation was written by Martin Langer at Ostfalia University. Netnod staff have created two NTS client implementations, one in Go, one in Python. Chrony has an experimental NTS branch.
For this HOWTO, we will focus mostly on how to use NTPsec since it's one of the first widely used NTP implementations with official NTS support.
This HOWTO assumes that you are using a Debian Linux based distribution. The instructions below have been tested on Debian 9 (Stretch), Debian 10 (Buster), Ubuntu 16.04 LTS (Xenial Xerus) and Ubuntu 18.04 LTS (Bionic Beaver). If you don't run Debian or Ubuntu on your computer you can download a virtual machine, or use a Docker container with Debian. This HOWTO also assumes that you have can become the root user using the “sudo” command.
Building and running ntsclient
ntsclient is an NTS client written in Go. It targets all *nix systems.
Using Go 1.11 or later, do:
git clone https://gitlab.com/hacklunch/ntsclient; cd ntsclient; make
On Linux if you don’t want to run ntsclient as root, do:
sudo setcap CAP_SYS_TIME+ep ./ntsclient
then run with:
./ntsclient --config ntsclient.toml
If you want to know what’s happening, add --debug:
./ntsclient --config ntsclient.toml
See https://gitlab.com/hacklunch/ntsclient for more information.
First you need to install the Debian "git" package that is needed to check out the NTPsec source code:
sudo apt-get update sudo apt-get install git
Next, check out the NTPsec source code:
git clone https://gitlab.com/NTPsec/ntpsec.git
Change directory to the one you just checked out:
The latest version of the master branch of NTPsec can occasionally break. If that happens, you can check out the version of NTPsec we used when writing this document:
git checkout 01eee1ed
NTPsec has a script which can install all the packages that are needed to build it. Run the script as root. Behind the curtains this will run apt-get to install everything:
We can now build NTPsec using the “waf” tool:
./waf configure ./waf build
Next we need a configuration file which tells NTPsec to use Netnod’s NTS servers. Create a file “ntp.conf” with the following contents:
# Exchange time with everybody, but don't allow configuration. # This is the right security setup for 99% of deployments. restrict default kod limited nomodify nopeer noquery restrict -6 default kod limited nomodify nopeer noquery # Local users may interrogate the NTP server more closely. restrict 127.0.0.1 restrict -6 ::1 # Minimal logging - we declare a drift file and that's it. driftfile /var/lib/ntp/ntp.drift server nts.ntp.se:3443 nts iburst server nts.sth1.ntp.se:3443 nts iburst server nts.sth2.ntp.se:3443 nts iburst
The first few lines starting with "restrict" and "driftfile" are good default settings for most NTPsec installations. The lines starting with "server" tells NTPsec to talk to Netnod's NTS servers on port 3443.
Update Apr 20, 2020: An incompatible change was made in the NTS draft as of version 26. For those using a NTS client supporting the new behaviour, port 3443 is to be used. Clients having the old behaviour should use port 4443. For more information see here
Testing NTPsec by hand
This section shows how you can try out NTPsec directly from the directory where it was built.
First of all, make sure that you are not running any other NTP server or client on your machine. The easiest way to do that is to stop the common NTP services:
sudo service ntp stop sudo service chrony stop sudo service openntpd stop
Warning: this will stop any time synchronisation you might have had installed on your computer.
Start the NTPsec server. The flags “-n” tells ntpd to run in the foreground, “-d” tells it to print debug information, “-c ntp.conf” tells it to use the configuration file we just created:
sudo ./build/main/ntpd/ntpd -n -d -c ntp.conf
You’re going to see lots of debug output. Look for some lines like this which shows that NTPsec is actually talking to Netnod’s NTS servers:
2019-11-06T14:01:04 ntpd: NTSc: NTS-KE req to nts.sth2.ntp.se:3443 took 0.073 sec, OK 2019-11-06T14:01:04 ntpd: DNS: dns_check: processing nts.sth2.ntp.se:3443, 1, 21901 2019-11-06T14:01:04 ntpd: DNS: Server taking: 18.104.22.168:4123 2019-11-06T14:01:04 ntpd: DNS: dns_take_status: nts.sth2.ntp.se:3443=>good, 0
Normally one doesn’t run NTPsec in debug mode. Instead it is started as a daemon in the background (a service is called a “daemon” on Linux) and a tool called “ntpq” is used to check on the status of NTPsec. To use the tool, open a new terminal, change directory to where you checked out and built the NTPsec source code, and run the following command.
./build/main/ntpclients/ntpq -p -u -w
“-p” tells ntpq to print information about the NTP servers NTPsec is talking to, “-u” tells it to show the units of any values it prints, and “-w” tells it to use a wide output format and not truncate the names of the NTP servers. The output should look something like this:
remote refid st t when poll reach delay offset jitter =============================================================================== *ntsts.sth.ntp.se 22.214.171.124 2 8 20 64 377 627.43us 5.487us 113.35us +ntsts.sth1.ntp.se 126.96.36.199 2 8 12 64 377 447.83us 56.455us 82.183us +ntsts.sth2.ntp.se 188.8.131.52 2 8 15 64 377 413.21us -88.19us 50.772us
“remote” is the name of the NTS server. Note that the name is “nts.sth.ntp.se”, not “nts.ntp.se” as specified in the configuration file. That is because the NTS protocol is split into two parts, first an encrypted TLS connection over TCP to a “NTS key establishment server”. When the client has received a set of keys, it will use a UDP protocol for the actual timestamping. Netnod runs the key establishment server and the timestamping server on different addresses.
The characters at the beginning of the server name says what NTPsec thinks about the server. An asterix (*) means that NTPsec has decided to use this server as its primary reference. A plus sign (+) means that NTPsec uses this server in its algorithm to estimate the current time. “refid” is the reference ID for the server, that is, where the NTS server gets its time from. If you look up these IP addresses you will see that they are for Netnod’s NTP servers: sth1.ntp.se and sth2.ntp.se. “st” means “stratum”, which is how far away from a primary clock source, such as an atomic clock, the NTP server is. sth1.ntp.se and sth2.ntp.se are stratum 1 servers; Netnod’s NTS servers talk to them and are stratum 2 servers. The rest of the line shows statistics about NTPsec’s current time estimate relative to the servers time estimate. To read a more detailed description of the output see the NTPsec manual page for the ntpq command.
If you are really quick at running ntpq after you started ntpd it will show “nts.ntp.se” as the remote and “.NTS.” as the refid. That means that NTPsec is in the process of connecting to the NTS key establishment server and has not yet been able to talk to the timestamping server.
If your computer has IPv6 connectivity, NTPsec might be talking to Netnod's NTS servers over IPv6. In that case, NTPsec will usually show the IP address in the "remote" field instead of a host name. That is quite normal.
Other NTS providers beside Netnod
The team behind NTPsec run two public NTS enabled NTPsec servers. You can configure NTPsec to talk to them by adding a "server" line with "ntp1.glypnod.com:123" (San Francisco) or "ntp2.glypnod.com:123" (London) to the ntp.conf file.
You can also configure NTPsec to talk to Cloudflare’s NTS servers by adding a “server” line with “time.cloudflare.com:1234” to the ntp.conf file.
Special note about TLS version
Please note that some providers of NTS, like Cloudflare, require TLS 1.3 support and that the version of OpenSSL (1.0.2) that comes with many Linux distributions (such as Debian 9 and Ubuntu 16.04) is too old to support TLS 1.3 and will not work with Cloudflare's servers. OpenSSL 1.1.1 that comes with Debian 10 or Ubuntu 18.04 is fine. Netnod and the NTPsec team have made the decision to support TLS 1.2 and newer so their NTS servers will work even if you run an older Linux distribution.
Installing NTPsec permanently
To install NTPsec permanently so that it starts as a daemon whenever your computer boots, do the following:
First uninstall any other NTP server you may have installed on your computer.
sudo apt-get remove ntp ntpsec chrony openntpd
Then install NTPsec systemwide. The default is to install NTPsec in /usr/local. Do that with:
sudo ./waf install
Copy the ntp.conf file to /etc and make sure that it is owned and only writable by root:
sudo cp ntp.conf /etc sudo chown root.root /etc/ntp.conf sudo chmod 644 /etc/ntp.conf
If you are still trying out NTPsec in debug mode, stop it by pressing Ctrl-C in that terminal. Then start the NTPsec daemon:
NTPsec will start itself in the background and return to the command line almost immediately.
Check that NTPsec is now running by using the "ntpq" command as shown above.
To make this permanent, edit /etc/rc.local and start NTPsec from there. The file should look something like this:
#! /bin/sh # there may be more lines here in the system's default rc.local /usr/local/sbin/ntpd exit 0
If rc.local doesn't exist on your computer, just create a new file. Also make sure that /etc/rc.local is executable:
sudo chmod +x /etc/rc.local
If you reboot your computer now, ntpsec should start automatically. Once more, check that it is running by using the "ntpq" command above.
There are more modern ways of starting a daemon than from rc.local, for example using a systemd unit file such as etc/ntp.service in the NTPsec source. But starting it from rc.local works on just about any old Linux distribution out there so in the name of simplicity that's what is described here.
The names of Netnod's NTP servers
Or unicast vs anycast.
All Netnod’s services are designed with a focus on redundancy and resilience. Consequently, our NTP service uses a distributed timescale with autonomous nodes throughout Sweden that can be reached in multiple ways.
We currently have four sites with NTP servers: Gothenburg, Malmö, Stockholm and Sundsvall. Each site has two Caesium Atomic Clocks and two NTP frontents. For example, the NTP servers in Stockholm are called sth1.ntp.se and sth2.ntp.se. Currently only the Stockholm site supports NTS but does have two redundant NTS servers: nts.sth1.ntp.se and nts.sth2.ntp.se.
Many simple NTP clients, for example the NTP client in common home routers, only allow the user to configure one NTP server. To cater for such devices Netnod also provides an anycast address for the NTP/NTS services: NTP at “ntp.se” and NTS at “nts.ntp.se”. This address is shared between all servers at all sites and a client which uses this address will be redirected to one of our sites. Exactly which site depends on where the user is located and which site is “closest” at that moment from a network topology viewpoint. This means that at lunch a client that sends a request to ntp.se might get a response from gbg2.ntp.se, while in the evening the same request might get a response from svl1.ntp.se. This is very robust. If the network would go down between the user and one site, the routing will be updated so that the user gets redirected to a server at another site.
Since the sites are in different locations relative to the user, the network delay varies, and the delays can be asymmetric: that is, it might take longer for a request from the client to reach the server than it takes for the response from the server to reach the client. This means that the time seen by a NTP client which uses ntp.se will vary. This isn’t that important if all the client needs is time accurate to a second, but can become an issue if the client needs time accurate to milliseconds. An NTP client which knows about the specific servers at each site will always talk to the same server and get more predictable network delays. It can also weigh the times received from each server and account for the different network delays and asymmetries to each site when calculating its estimate of the current time.
Summary: If you really care about time you should use a NTP/NTS client which supports multiple NTP servers. You should put as many servers as possible (within reason) in your configuration file to provide the NTP client with as much information as possible about each server and to ensure robustness against network and server failure. If you just want only “approximate” time and the robustness anycast provides, or if your client only supports one NTP server, use the anycast address “ntp.se”, or for NTS “nts.ntp.se”.