I recently had to redo the setup for Google Assistant. Everything was correct (URLs in Google, ports, port forwarding, certs, etc.) but the process was failing when trying to connect with Google Assistant via the app to add the devices. While doing many troubleshooting steps, after some time, it would randomly work once, then not again for some time. More troubleshooting with failures, then randomly work. I had updated my DNS about 12-15 hours prior, which I thought was enough time, but it looks like it needed longer. Simply waiting was sufficient. I am hoping that this tip might save others some time pulling their hair out. :)