When a TLS connection is established, the server will present a certificate, and possibly intermediate certificates.
Your device should then decide if it trusts the presented certificate. It does so by having a trust store, where it holds certificates it… trusts.
A certificate presented by the sever may be trusted if:
- The certificate itself is trusted. This is how you would be able to work with self-signed certificates, for instance.
- One can establish a certificate chain from that certificate to a certificate which is trusted. This is the most common case.
In the latter case, the device will check who signed the certificate (it will give the identity of another certificate). It will try to find that certificate, either in the list of certificates presented by the server, or in the trust store.
If it doesn’t find the signing certificate, then the original certificate cannot be trusted.
Then it will check that the signature on the original certificate matches the signing certificate. It it doesn’t, again, the certificate won’t be trusted.
[skipping a number of other checks such as validity dates, hash types and sizes, etc.]
Now we know that the original certificate (A) was signed by another certificate (B). If B was in the trust store (more specifically, trusted to sign other certificates), then that’s it, since A is signed by B and we trust B, then we trust A.
If B was not in the trust store but in the list of certificates presented by the server (an intermediate cert), then we start the process again, trying to find who signed this cert, whether we can trust it, etc.
There are different ways of establishing what certificates your server will trust.
The normal way in a public PKI, like that used to check domain names in browsers, is to have a (looooong) list of “root certificates” in the trust store. Each vendor, independently and/or in cooperation with the CA/Browser forum will decide which Certification Authorities (CAs) they trust, and thus which root certificates to include.
That list is frequently updated, with new root certs added, old ones replaced, and sometimes certs removed altogether (when the device vendor and/or industry consider that the CA is not trustworthy and has issued certificates incorrectly, or is at risk of doing so).
In some cases the private key of a cert may be leaked. In such cases the associated certificate can no longer be trusted.
Also remember that when you renew your certificate, there is absolutely no guarantee that it will still be signed by the same chain of certificates, or even the same root.
So the public PKI is a living thing. You just can’t rely on a small and fixed list of root certs if you use it.
If your device has the ability to hold a full certificate store, you should have that, and include a way to keep it updated. You can for instance find curl’s CA list, extracted from Mozilla, here.
The alternative is not to use the public PKI, but to have your own CA and/or certs.
Some people will just store the server cert directly in the device. Others will set up they own CA, store the root cert of the CA in the device, and issue certificates signed by that CA for their server.
In any case, it’s still a very good idea to be able to:
- Store multiple trusted certs
- Have a way to update that list
The mechanism will be exactly identical to the public PKI case. The differences will be that:
- The list of trusted certs will be a lot shorter, so on very constrained devices that can help.
- The list of trusted certs should be updated a lot less often. Again, that could help for some devices with limited connectivity.
The fact that your server is also accessible by browsers is not an issue: just point two different names to your server, and use different certificates, one from the public PKI and another from your own. Use the domain with the public PKI cert for generic users, the other for your devices.
However this would fail if your devices end up behind a proxy or firewall (explicit or transparent).