Iris

Our project for the NUR festival is an interactive installation that unveils the unique beauty of the human eye. A guest takes a photo, and a moment later, they see their iris on a giant screen, enveloped in a cloud of meditative visual effects. The main challenge was managing the constant flow of people under highly volatile shooting conditions, with unpredictable glares and harsh lighting. To keep the queue moving, we had just 15 seconds for everything: from the click of the shutter to the final visualization.

At the core of the setup is a continuously running Python server. Upon startup, it loads all neural network models into memory just once, and then simply monitors a specific folder where the photographer drops new shots in real time.

Each image goes through a three-stage recognition pipeline, and the first method to succeed wins:

• MediaPipe Face Mesh – used for standard portraits where the full face is visible; iris keypoints are extracted directly. However, this didn't quite suit our needs as the iris ended up being too low-resolution.
• SAM (Segment Anything) on the full frame – used for close-ups of the eye without the face. Among the detected segments, the system identifies the dark pupil and a matching iris around it based on size and contrast.
• BlazeFace + SAM on a cropped region around the eye – a backup method for when the face is only partially visible.

Next, a series of validation checks filters out false positives caused by glare, the sclera, and eyelids.

Once validated, a square containing the iris is cropped from the source image. Glares are removed using neural network inpainting (LaMa), the pupil is resized and perfectly centered, and the brightness and color are normalized. The final output is a clean iris on a transparent background with soft edges.

This processed frame is then sent to TouchDesigner, where the iris appears on the big screen in real time, enhanced with my custom visual effects.

Over the three days of the NUR festival, 1800 guests went through the installation.

As a byproduct, I compiled an anonymous, labeled dataset from these photos and trained a custom model to detect the iris and pupil. After optimization, it ended up working roughly a thousand times faster than the previous pipeline — achieving around 30 frames per second, which means it now runs in true real-time.