41 behaviours and 94 object classes: what IRIS knows how to see

This site is full of rules written as a single sentence: ‘tell me if someone walks into dock 4 at night’. Underneath that sentence there is always one of these 41 behaviours with its parameters filled in. Presence in a zone, leaving a zone, occupancy limits, line crossing, tailgating, dwell time continuous or accumulated, loitering, long path, abandoned object, removed object, bay taken or free, wrong way, sudden acceleration, queue, crowding with density and risk, distance between people, sequence of zones. None of them is custom-coded for one customer: they are engine blocks, with their units and factory values, that combine with each other. That is why what we demonstrate in one area of this site works the same in another, and why a new case is not a development project: it is one more rule. They are counted and published one by one, which is the only way anyone can check whether theirs is on the list.

A detector says ‘car’; what matters is that the whole system means the same thing by ‘car’. IRIS keeps its own catalogue of 94 classes across 12 families — people, vehicles, animals, accessories, street furniture, furniture, electronics, food — and every detection model translates its own vocabulary into that catalogue. It is hierarchical: ticking ‘Vehicle’ counts car, lorry, bus and motorbike without naming them one by one, and you can also ask for the family alone. The names are written in the eight languages of this site. What the customer gains is concrete: a filter set up today keeps working the day the model underneath is swapped for a better one. And above the class there is a second level of detail: a car is refined into saloon, SUV, van, lorry, bus, motorbike, bicycle, emergency vehicle, motorhome or special vehicle.

Here is the difference between a detection and a piece of data. Something moving fast in the picture means nothing on its own: a distant lorry looks slow and a nearby bird looks like a missile. To be able to say how many kilometres per hour that vehicle is doing, somebody has to have explained to the camera what the ground it is looking at is like. There are three routes, in order of effort: with nothing done at all, everything is measured in heights of the object itself, comparable within the same scene; with the average height of each class, which ships already filled in, you get metres per second and kilometres per hour; and by marking four points on the ground with their real measurements, you measure for real, in metres on the ground plane. And it always says which one was used: estimated speed is labelled as estimated, a rule can demand that only the measured value counts, and if the calibration is not valid the system drops back to uncalibrated units instead of inventing a metre. For the same reason, the points of a track are not joined by a line: between two points a few seconds apart nobody measured anything, and the line would claim a path that does not exist.

Anyone who has lived with video analytics before has the same fear: that it will beep all night. So every behaviour filters by class, by confidence, by size, by how long the object has been tracked and by which point of the box decides — the feet, the centre, the whole box, or any part that grazes it. There are areas where nothing is detected on purpose, a double threshold so the alert does not flicker at the edge of a zone, and four controls for waiting, cooling down and capping alerts per minute. Before a rule is published you see on the live image what it really produces, including the objects that fail the filter and the reason why, and you can simulate it without it raising a single alert. And when something cannot be checked, the object does not pass and the reason is stated: a gap is never filled with an invented value. What it does not do is stated too: IRIS does not know who anyone is — it compares the appearance of an object, not the identity of a person — and where the input needed to give metres is missing, the field is not published at all: absent, never estimated. There are 23 guided recipes for the usual rules, and everything travels over a single secure port: the network team gets asked for one firewall rule, not thirteen.

An integrator is not looking for a feature: they are looking for theirs, which is why the 41 are grouped by the question they answer. Nine ask about a zone — who came in, who left, how many are inside, how many is too many; five about where an object has been — where it appeared, whether it passed through here, whether it ended up in there, whether it did A and then B; four about a line and its arrow; four about the time spent inside; four about how it moves; and four about whether one thing is inside another: a person in a vehicle, an item on a pallet, an empty pallet. The remaining eleven cover what appears and what disappears, crowds, distance between objects, parking bays and the quality of what was detected. If the case is none of the 41, you build it: the fine filter compares each object against eleven fields with eight comparators, and conditions are grouped with ‘and’ and ‘or’ until the rule says exactly what it has to say. Geometry is not redrawn either: a line drawn once is applied to twenty identical doors at a stroke, and it is shared as it stands or copied to be tweaked camera by camera. And the wizard hands back the whole rule summarised in a single sentence, in the reader's own language, so that whoever signs it understands what they signed.

That an object stays the same object while it crosses the scene sounds like a technical detail, and it is the ground everything else stands on. Dwell time, path and speed are three ways of saying ‘this same one’: if the identity breaks every time somebody walks behind a pillar, all three turn into noise. IRIS holds the identity while the object moves, keeps it for a few seconds when something hides it and recovers it when it comes back; and each object keeps a small history of its classes, so a car that flickers to lorry in the distance does not make the rule flicker with it. Out of that comes what stays written about each object: over forty facts — class and sub-type, colour across eleven families, size, heading, acceleration, time in each zone, lines crossed with their timestamp, who it was with — and with them you can reconstruct months later where it went, how long it lasted, how fast it was going and which recording covers that instant. Every fact travels with its origin and its time: which camera produced it and in which time zone, so a report holds up in front of whoever disputes it. And while an object is still alive its life summary does not exist yet: that is a gap with a reason, not a failure, and it is shown as such.