41 behaviors and 94 object classes: what IRIS knows how to see

This site is full of rules written as a single sentence: “tell me if someone walks into dock 4 at night.” Underneath that sentence there is always one of these 41 behaviors with its parameters filled in. Presence in a zone, leaving a zone, occupancy limits, line crossing, tailgating, dwell time continuous or accumulated, loitering, long path, abandoned object, removed object, space taken or free, wrong way, sudden acceleration, queue, crowding with density and risk, distance between people, sequence of zones. None of them is custom-coded for one customer: they’re engine blocks, with their units and factory values, that combine with each other. That’s why what we demonstrate in one area of this site works the same in another, and why a new case isn’t a development project: it’s one more rule. They’re counted and published one by one, which is the only way anyone can check whether theirs is on the list.

A detector says “car”; what matters is that the whole system means the same thing by “car.” IRIS keeps its own catalog of 94 classes across 12 families — people, vehicles, animals, accessories, street furniture, furniture, electronics, food — and every detection model translates its own vocabulary into that catalog. It’s hierarchical: ticking “Vehicle” counts car, truck, bus and motorcycle without naming them one by one, and you can also ask for the family alone. The names are written in the eight languages of this site. What the customer gains is concrete: a filter set up today keeps working the day the model underneath is swapped for a better one. And above the class there is a second level of detail: a car is refined into sedan, SUV, van, truck, bus, motorcycle, bicycle, emergency vehicle, motorhome or special vehicle.

Here’s the difference between a detection and a piece of data. Something moving fast in the picture means nothing on its own: a distant truck looks slow and a nearby bird looks like a missile. To be able to say how many miles per hour that vehicle is doing, somebody has to have explained to the camera what the ground it is looking at is like. There are three routes, in order of effort: with nothing done at all, everything’s measured in heights of the object itself, comparable within the same scene; with the average height of each class, which ships already filled in, you get feet per second and miles per hour; and by marking four points on the ground with their real measurements, you measure for real, in feet on the ground plane. And it always says which one was used: estimated speed is labeled as estimated, a rule can demand that only the measured value counts, and if the calibration isn’t valid the system drops back to uncalibrated units instead of inventing three feet. For the same reason, the points of a track aren’t joined by a line: between two points a few seconds apart nobody measured anything, and the line would claim a path that doesn’t exist.

Anyone who’s lived with video analytics before has the same fear: that it’ll beep all night. So every behavior filters by class, by confidence, by size, by how long the object has been tracked and by which point of the box decides — the feet, the center, the whole box, or any part that grazes it. There are areas where nothing’s detected on purpose, a double threshold so the alert doesn’t flicker at the edge of a zone, and four controls for waiting, cooling down and capping alerts per minute. Before a rule is published you see on the live image what it really produces, including the objects that fail the filter and the reason why, and you can simulate it without it raising a single alert. And when something can’t be checked, the object doesn’t pass and the reason is stated: a gap is never filled with an invented value. What it does not do is stated too: IRIS does not know who anyone is — it compares the appearance of an object, not the identity of a person — and where the input needed to give feet is missing, the field is not published at all: absent, never estimated. There are 23 guided recipes for the usual rules, and everything travels over a single secure port: the network team gets asked for one firewall rule, not thirteen.

An integrator isn’t looking for a feature: they’re looking for theirs, which is why the 41 are grouped by the question they answer. Nine ask about a zone — who came in, who left, how many are inside, how many is too many; five about where an object has been — where it appeared, whether it passed through here, whether it ended up in there, whether it did A and then B; four about a line and its arrow; four about the time spent inside; four about how it moves; and four about whether one thing is inside another: a person in a vehicle, an item on a pallet, an empty pallet. The remaining eleven cover what appears and what disappears, crowds, distance between objects, parking spaces and the quality of what was detected. If the case is none of the 41, you build it: the fine filter compares each object against eleven fields with eight comparators, and conditions are grouped with “and” and “or” until the rule says exactly what it has to say. Geometry isn’t redrawn either: a line drawn once is applied to twenty identical doors at a stroke, and it’s shared as it stands or copied to be tweaked camera by camera. And the wizard hands back the whole rule summarized in a single sentence, in the reader’s own language, so that whoever signs it understands what they signed.

That an object stays the same object while it crosses the scene sounds like a technical detail, and it’s the ground everything else stands on. Dwell time, path and speed are three ways of saying “this same one”: if the identity breaks every time somebody walks behind a pillar, all three turn into noise. IRIS holds the identity while the object moves, keeps it for a few seconds when something hides it and recovers it when it comes back; and each object keeps a small history of its classes, so a car that flickers to truck in the distance doesn’t make the rule flicker with it. Out of that comes what stays written about each object: over forty facts — class and sub-type, color across eleven families, size, heading, acceleration, time in each zone, lines crossed with their timestamp, who it was with — and with them you can reconstruct months later where it went, how long it lasted, how fast it was going and which recording covers that instant. Every fact travels with its origin and its time: which camera produced it and in which time zone, so a report holds up in front of whoever disputes it. And while an object is still alive its life summary doesn’t exist yet: that’s a gap with a reason, not a failure, and it’s shown as such.