Skip to content

FaceStream.AI

Documentation

Integrations

Face recognition in Home Assistant

Face recognition for Home Assistant via MQTT discovery: a device per camera, automations for arrivals, strangers and the doorbell, a live dashboard.

Applies from version 2.4.0

On this page

FaceStream.AI brings its cameras into Home Assistant by itself. Every camera becomes a device: what it recognises, who arrived last and when, whether somebody is standing there right now, the face of the last arrival and a button to start a recognition. No YAML, no templates, no custom integration. It works through MQTT discovery, so Home Assistant needs its MQTT integration and a broker that both can reach.

FaceStream.AI reports; your automations in Home Assistant decide what happens next, a welcome scene, a message on your phone, the hallway light, the door.


What you need

FaceStream.AI Pro and up, the MQTT connection is part of Pro. The Recognise now button also needs Recognition on request, which Pro includes
Home Assistant 2024.11 or later, with the MQTT integration set up
An MQTT broker Reachable from Home Assistant and from FaceStream.AI. On Home Assistant OS, the official Mosquitto broker add-on is the easiest: once it runs, Home Assistant offers to set up the MQTT integration by itself
A broker user for FaceStream.AI With the Mosquitto add-on, create a Home Assistant user, for example facestream, and use its name and password, the add-on accepts Home Assistant users

Switch it on

  1. In FaceStream.AI, open Notifications and, under Connections, open the MQTT connection to the broker Home Assistant uses, or Add connection and choose MQTT.
  2. Broker is the address of the broker, with the Mosquitto add-on, the address of Home Assistant, for example 192.168.1.30. Port 1883, User and Password as created above.
  3. Tick Home Assistant discovery. Leave Discovery prefix at homeassistant unless you changed it in Home Assistant.
  4. Save connections and rules.

An MQTT connection with Home Assistant discovery ticked: broker, port, user, topic, QoS, message, discovery prefix

A few seconds later Home Assistant lists one device per camera under Settings → Devices & services → MQTT. Nothing needs restarting.

Home Assistant lists the FaceStream.AI devices under MQTT: one per camera, plus the installation itself

Topic and Message are for rules of your own; discovery does not use them. A rule is not needed either: every event of every camera reaches Home Assistant, whatever your rules say, and rules keep working as before for your own topics and wording.


What appears

Each camera is one device, named like the camera, so it can go into the room where it hangs.

Entity Type What it shows
Recognition Event Every event of this camera. The event type is visit_started, visit_ended, unmatched, trigger or spoof, the same events a rule can react to. Its attributes carry name, known, camera, camera_id, liveness, duration, sightings, at, image_url and frame_url
Visit in progress Binary sensor Whether somebody is at this camera right now. On from the moment somebody arrives, off when the visit ends
Last person Sensor Who arrived last: a name, or Unknown. Its attributes carry at, known, liveness, camera, camera_id, image_url and frame_url
Last seen Sensor When that was, as a timestamp
Last face Image The face photo of that arrival
Recognise now Button Starts a recognition at this camera, with the camera's own Look for, Attempts and Stop once recognised

Home Assistant names the entities after the camera and the entity, so a camera called Front Door gives event.front_door_recognition, binary_sensor.front_door_visit_in_progress, sensor.front_door_last_person, sensor.front_door_last_seen, image.front_door_last_face and button.front_door_recognise_now. The settings of each entity show the exact ID; the examples below use these.

Last person, Last seen and Last face change when somebody arrives, a visit starting, or a face nobody could place. They stay as they are when the visit ends, who came last does not change by their leaving. What changes then is Visit in progress. A suspected spoof and a request show up as events only.

All of them survive a restart, of Home Assistant and of FaceStream.AI. Visit in progress is answered fresh after a restart, from whether a visit is actually open, so it is never left standing on the wrong state.

The Front Door device in Home Assistant: the Recognise now button, the last face, the last person, when they were seen and whether a visit is in progress

When FaceStream.AI stops, or the broker loses it, its cameras show as unavailable.

FaceStream.AI itself

Besides the cameras there is one more device, named FaceStream.AI. It stands for the installation rather than for any one camera, and it carries what belongs to the whole program:

Entity Type What it shows
Update Update Whether a newer version is available, with its release notes

The FaceStream.AI device in Home Assistant, reporting that a newer version is available

update.facestream_ai_update appears on Home Assistant's update page next to everything else in the house. It reports and offers no install button: an update stops recognition for a moment and replaces the image, so it stays a decision for whoever runs the installation, not for an automation.

It offers the version your licence may install. If a newer one exists that your update period no longer covers, it is not offered here, FaceStream.AI's own Overview says so and how to extend. With the update check switched off there is no Update entity, rather than a claim that everything is current.

One camera that stops delivering, unplugged, rebooting, a stream that drops away, goes unavailable on its own, shortly after its stream stops answering: four to fifteen seconds in our tests, depending on where the camera had got to. The other cameras carry on, and so does everything else FaceStream does. Recognise now cannot be pressed for a camera in that state: the request would reach nothing. Once the camera delivers again, it comes back within a second or two.


Automations

Every automation below reacts to the Recognition event entity and then looks at the event type and the attributes. Paste them in Settings → Automations & scenes → Create automation → Edit in YAML, and replace the entity IDs with yours.

not_from: unavailable keeps an automation from firing when a camera comes back after being unavailable, that, too, changes the entity's state.

Welcome a family member home

alias: Welcome home, Amelia
triggers:
  - trigger: state
    entity_id: event.front_door_recognition
    not_from: unavailable
conditions:
  - condition: template
    value_template: >-
      {{ trigger.to_state.attributes.event_type == 'visit_started'
         and trigger.to_state.attributes.name == 'Amelia Hart' }}
actions:
  - action: scene.turn_on
    target:
      entity_id: scene.hallway_welcome
mode: single

For several people, compare with a list: trigger.to_state.attributes.name in ['Amelia Hart', 'Tom Hart'].

A message with the photo when a stranger is at the door

At a camera set to On request only, a stranger arrives as unmatched; at a camera that recognises continuously, as visit_started with known false. This covers both:

alias: Stranger at the front door
triggers:
  - trigger: state
    entity_id: event.front_door_recognition
    not_from: unavailable
conditions:
  - condition: template
    value_template: >-
      {% set e = trigger.to_state.attributes %}
      {{ e.event_type == 'unmatched'
         or (e.event_type == 'visit_started' and not e.known) }}
actions:
  - action: notify.mobile_app_your_phone
    data:
      title: Somebody at the front door
      message: Not recognised.
      data:
        image: "{{ trigger.to_state.attributes.frame_url or trigger.to_state.attributes.image_url }}"
mode: single

frame_url is the whole picture, image_url the face. Both open without signing in, but your phone has to be able to reach FaceStream.AI's address, at home, or over a VPN.

Light while somebody is at the door

Visit in progress is a state, not an event, so it can be waited on and asked about. It is on from the arrival until the visit ends, a visit ending when nobody has been seen for as long as the sightings belong to the same visit setting says.

alias: Hallway light while somebody is at the front door
triggers:
  - trigger: state
    entity_id: binary_sensor.front_door_visit_in_progress
    not_from: unavailable
actions:
  - if:
      - condition: state
        entity_id: binary_sensor.front_door_visit_in_progress
        state: "on"
    then:
      - action: light.turn_on
        target:
          entity_id: light.hallway
    else:
      - action: light.turn_off
        target:
          entity_id: light.hallway
mode: single

The same entity works as a condition elsewhere, only ring the chime if nobody is already standing there, or on a dashboard as a badge that says whether the door is busy.

Recognise whoever rings the bell

For a camera set to On request only: when the doorbell is pressed, Home Assistant presses Recognise now. Any doorbell Home Assistant knows about will do.

alias: Recognise whoever rings
triggers:
  - trigger: state
    entity_id: binary_sensor.front_door_bell
    to: "on"
actions:
  - action: button.press
    target:
      entity_id: button.front_door_recognise_now
mode: single

The answer arrives a moment later as an event on event.front_door_recognition: a name if the camera recognised somebody, unmatched if it saw a face and could not place it, nothing if nobody was there.

Opening a door

FaceStream.AI does not open doors; an automation of yours might. If it does, let it act only on a visit_started of named people, only when the liveness check ran, liveness is empty when it did not, and only right after somebody rang:

alias: Let the family in
triggers:
  - trigger: state
    entity_id: event.front_door_recognition
    not_from: unavailable
conditions:
  - condition: template
    value_template: >-
      {% set e = trigger.to_state.attributes %}
      {{ e.event_type == 'visit_started'
         and e.name in ['Amelia Hart', 'Tom Hart']
         and e.liveness is not none }}
  - condition: template
    value_template: >-
      {{ (now() - states.binary_sensor.front_door_bell.last_changed).total_seconds() < 30 }}
actions:
  - action: lock.unlock
    target:
      entity_id: lock.front_door
mode: single

Checking event_type matters: a suspected spoof carries the name of the person in the photo, and an automation that only looks at name would let a printed photo in. The liveness check stops a photo held up to the camera on the off chance; it does not stop somebody who prepared. Decide for yourself whether a face may be the only key to your door. Rules that open a door says more.


The live picture on a dashboard

Home Assistant can show FaceStream.AI's annotated picture, faces framed and named, like any other camera:

  1. In FaceStream.AI, open the camera's Trigger dialog and copy the Stream URL for other systems, http://192.168.1.20:8100/stream/front-door?token=….
  2. In Home Assistant, Settings → Devices & services → Add integration → MJPEG IP Camera, and paste it as the MJPEG URL. Leave the still image URL, user and password empty; the token in the address is the sign-in.
  3. Put the new camera entity on a dashboard with a picture entity card.

Next to it, a picture entity card for image.front_door_last_face with sensor.front_door_last_person and sensor.front_door_last_seen answers who was it, and when? at a glance, and binary_sensor.front_door_visit_in_progress says whether they are still there.

The face photo

Home Assistant fetches the photo from FaceStream.AI itself, so the address in front of it has to be one Home Assistant can reach.

Notifications → Address for image links holds it. Leave it empty and FaceStream.AI follows whatever address you open its interface under, which is right in most houses. Fill it in when that is not the address others reach it by: behind a reverse proxy, under a name of its own, or when you work on the interface from somewhere else. What you enter stays, a later visit to the interface from another address no longer overwrites it.

Below the field you see what a photo link looks like with the current setting, and a word when the address cannot work, localhost and 127.0.0.1 point every receiver at itself, not at FaceStream.AI.

The link opens without signing in: anyone who has it can see that one photo, and it cannot be guessed from a name or a time.

Names and photo links are not kept on the broker. Only the device descriptions and the availability are, so the broker never holds on to who came last. What Home Assistant needs after a restart, FaceStream.AI sends again when Home Assistant announces itself.


Topics

For anyone who wants to use them outside Home Assistant. <installation> is twelve hexadecimal digits that stay the same for this installation; <camera> is the camera id, with every character other than a letter, a digit, _ or - replaced by _.

Topic Direction Payload
homeassistant/device/facestream_<installation>/<camera>/config to Home Assistant, retained the device and its entities
facestream/<installation>/status to Home Assistant, retained online or offline for the installation
facestream/<installation>/<camera>/status to Home Assistant, retained online or offline for this one camera
facestream/<installation>/<camera>/event to Home Assistant JSON with event_type and the attributes above
facestream/<installation>/<camera>/person to Home Assistant the name
facestream/<installation>/<camera>/face to Home Assistant the photo link
facestream/<installation>/<camera>/request to FaceStream PRESS starts a recognition

Messages go out with the QoS of the connection.


Who can press the button

Anyone who can publish to the broker can start a recognition, just like anyone who holds a camera's trigger token. It starts a recognition and nothing else. Give the broker users and passwords, and FaceStream an account of its own.


When something does not show up

What you see What to check
No FaceStream device under MQTT Is Home Assistant discovery ticked and saved? Do FaceStream.AI and Home Assistant use the same broker? Does the Discovery prefix match Home Assistant's? Notifications → Recent deliveries shows not connected to the broker when the broker refuses FaceStream.AI's sign-in or cannot be reached
The devices are there, but unavailable FaceStream.AI is stopped, has lost the broker, or that camera delivers no picture, check its card on the Overview
Last face stays empty Open the FaceStream.AI interface once by an address Home Assistant can reach, see The face photo
Your own rules do not change what Home Assistant gets They are not meant to: discovery sends every event. Filter in the automation

Home Assistant is a trademark of the Open Home Foundation. FaceStream.AI connects to it through the standard MQTT integration and is not affiliated with or endorsed by the Open Home Foundation.