Well – Since Kubernetes was a bust – I figured I should start trying to do some of the video analysis of my network locally. I got a Google Coral USB Accelerator – it’s basically a co-processor that adds and edge based TPU (Tensor Processing Unit) all at the end of a USB cord.
Starting out, I made an Ubuntu Server VM with 15gb of Ram and 3 cores. This will eventually host Frigate as well as a minimal HomeAssistant installation (which will send my main HA installation alerts over MQTT). DON’T DO THIS – for multiple reasons. As I found out later on, this was not the correct course of action – there is often an issue with how USB is passed to the VM, so it causes the Coral to freeze. I should have used an LXC (linux container) in Proxmox. Even better would have been installing ubuntu server directly on bare metal. In either case (LXC/Bare metal), it solves the problem. Also – there’s no need for a secondary HA installation – Frigate can send MQTT to a broker, which can send them to the main Home Assistant instance. I eventually put the MQTT broker on the same machine as the Frigate installation.
I started over with Ubuntu Server 20.04 on bare-metal, and then got going. (Using 22.04 caused some issues with the python version – so I preferred to install 20.04 and then upgrade everything later, aside from python)
I installed openssh and then started following the Coral tutorial:
sudo apt install openssh-server openssh-client
https://coral.ai/docs/accelerator/get-started/
There is one step that requires unplugging and re-plugging the Coral USB – I wasn’t near the machine so I simply restarted. This did not work. I needed to unplug the device -this worked.
Next step is to install Docker. I followed the instructions here to set up the apt repository and install the Docker Engine:
https://docs.docker.com/engine/install/ubuntu/
Docker compose (v2) should be included with this, but make sure by trying the following:
$ sudo apt-get update
$ sudo apt-get install docker-compose-plugin
Next is to set up Frigate – I struggled a little bit on this step because a few different places talked about writing a config file before spinning up the Docker container. I don’t think Frigate’s own instructions are super clear about this either.
https://docs.frigate.video/frigate/installation/#docker
I’ll spell this out just in case anyone else got tripped up here.
In my home directory, I made a folder called ‘frigate’ using the mkdir command. So when I use the print working directory command, it looks like ‘/home/myuser/frigate’ – with ‘myuser’ being replaced by whatever your user directory is. In here, I ran the command ‘nano docker-compose.yml’ which opens a blank yml file. From here, I pasted in the docker compose example on the frigate docs page from above. There are two lines that need to be changed:
– /path/to/your/config.yml:/config/config.yml
– /path/to/your/storage:/media/frigate
In mine, they look like this
– /home/myuser/frigate/config.yml:/config/config.yml
– /home/myuser/frigate/storage:/media/frigate
The left side of the : means where it is on the VM, and the right side is where it is inside the Docker container. Just make sure the ports are correct (I needed to add a port for my specific cameras), and then save it.
Also – I made sure to include the line in the ports:
– “544:544”
I’m using rtsp, and it all comes over port 544, so it’s included just to make sure it gets into the container properly.
Before moving on, make sure all the cameras are on the network and working with static IP addresses. I’m using reolink cameras, and I was able to set that up through the app. I then checked the rtsp streams with VLC to make sure it’s correctly streaming.
Next, I made another file in that folder called config.yml and this will eventually contain all of the information about our cameras, so let’s add that now. I found a few templates but the one I used I’ll have at the bottom of this post. Frigate also has a few on their site too:
https://docs.frigate.video/guides/getting_started
As I was adding the rtsp link in the config file, I was checking to make sure each of the streams was reachable by VLC. Interestingly enough, only one of my cameras worked initially. I had to go into each of the cameras’ settings and allow rtsp streaming. Also, I set up detection to use the sub streams instead of the main stream to reduce the processing overhead on the cpu/tpu.
One funny issue is that I thought in the yml file, the name of the camera needed to be a value – like this “name_of_your_camera: indoor1zoom” – but instead it should just be “indoor1zoom:” – my bad.
OK – so now, in the folder where the docker-compose.yml file is, just run “sudo docker compose up” and it should start the container. From there, you can go to http://[ip/FQDN]:5000 and you should see the Frigate page.
So at this point, I have a pretty nice setup. I already have an NVR for my cameras, so all i need Frigate to do is alert for detections – and if that’s integrated with Home Assistant (HA), I can get those alerts on my phone wherever I am.
At this point. I decided to redo everything, haha. I keep getting errors with the TPU, and it constantly hangs and restarts. There is apparently some issue with how USB is passed to the VM. It seems that using a LXC in Proxmox (linux container) is the way to go from the start. Even better is putting linux on the bare metal. So here, I made backups of all the config files, and then installed Ubuntu server.
Now with a fresh install of Ubuntu Server 20.04, the TPU errors seem to be going away – that said, it’s not actually detecting anything. It’s important to make sure the config file has all the right settings – recording needs to be enabled, objects need to be tracked, and cameras set up properly with the correct roles. Tweaking this will eventually give you an NVR that works really effectively.
The next step here is getting all of this over to Home Assistant. Step 1 here is to use HACS to download the Frigate integration of HA. Then after a restart, I added the integration (Home Assistant > Settings > Devices & Services > Add Integration > Frigate). After that, i plugged in the FQDN and it came right up! The camera feeds work well – but the actual alerts are still not working – so I believe it’s time for the final step – getting MQTT working.
To get all of those nifty alerts going, I need an MQTT Broker. I installed Mosquitto on the same machine as Frigate.
At first I struggled getting the Mosquitto config file properly working – the main issue is that I believe the mosquitto.conf file was in my home directory, however in truth, it existed in the /etc directory. Once I learned that, it still wasn’t easy. After screwing a bunch of stuff up, i had to remove mosquitto with the purge command and start fresh.
Eventually I found this guide, and it was very helpful:
https://www.vultr.com/docs/install-mosquitto-mqtt-broker-on-ubuntu-20-04-server/
I followed this very carefully, and had considerably more success than previously. It was here I went back into HomeAssistant, and was able to see MQTT in Settings>Integrations. I just put the server name in with the user and pass I just set up, and it worked great.
The last piece of it – just make sure all the MQTT items in the Frigate config file are correct – I had a typo on the IP address which prevented all of the sensor information from populating in HA. Once that was fixed, it’s all up and ready to go.
What a journey. That took me the better part of a week to get working.
I’ll include some helpful templates below.
config.yml template:
from https://github.com/blakeblackshear/frigate/discussions/1837
________________________________________
# ~/docker/frigate/config.yml
cat ~/docker/frigate/config.yml
mqtt:
host: 192.168.1.11
user: evan
password: secretpassword
objects:
track:
– person
– dog
– cat
snapshots:
enabled: True
bounding_box: True
detect:
width: 640
height: 360
fps: 5
detectors:
coral1:
type: edgetpu
device: pci:0
coral2:
type: edgetpu
device: pci:1
record:
enabled: True
retain:
days: 2
mode: active_objects
events:
retain:
default: 3
mode: active_objects
ffmpeg:
hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
cameras:
driveway_cam_frigate:
objects:
track:
– person
– car
– dog
– cat
– motorcycle
– bicycle
ffmpeg:
inputs:
– path: rtsp://evan:secretpassword@192.168.1.31:554/h264Preview_01_main
roles:
– record
– rtmp
– path: rtsp://evan:secretpassword@192.168.1.31:554/h264Preview_01_sub
roles:
– detect
front_door_cam_frigate:
ffmpeg:
inputs:
– path: rtsp://evan:secretpassword@192.168.1.32:554/h264Preview_01_main
roles:
– record
– rtmp
– path: rtsp://evan:secretpassword@192.168.1.32:554/h264Preview_01_sub
roles:
– detect
back_cam_frigate:
ffmpeg:
inputs:
– path: rtsp://evan:secretpassword@192.168.1.33:554/h264Preview_01_main
roles:
– record
– rtmp
– path: rtsp://evan:secretpassword@192.168.1.33:554/h264Preview_01_sub
roles:
– detect
rtmp:
enabled: False
live:
height: 1080
quality: 1