Implementation of DeepML Technologies for Anomalous Behavior Analysis and Detection of Unauthorized Objects

programming languages were used to develop the solution

100+

hours of footage were used to test PoC

80%

– up to this percentage, the operator response time is reduced

Description

A US-based international technology company reached out to Softacom. They are a global leader in access control, time management, and security systems. The company asked us to improve and automate their security system. They wanted to add features for detecting unauthorized objects and spotting abnormal human behavior in real time.

Softacom was chosen for this task because of our experience in AI and Machine Learning.

Detection of unauthorized objects refers to identifying large items that appear in places where they were not previously present. For example, a suitcase left at a train station (up to 50x50x50 cm in size) for more than an hour would be classified as an unauthorized object. However, something like a placed flower vase would not fall under this category.

As a definition for abnormal behavior, together with the client, we defined it as a prolonged presence of a person in areas of interest specified to us, such as unused spaces or access points to “hidden” zones.

Using the new functionality, it should be possible to mark and configure zones that will later be analyzed by the system operator. For example, if we have a warehouse with a marked zone, and a person remains in this zone for an extended period, the system sends a notification to the operator for further analysis.

Our task was to develop a PoC (Proof of Concept) solution – an application based on .NET and Python scripts. The .NET application serves as an interface for displaying the video footage provided by the client from their surveillance cameras, and for flexible system configuration (zones of interest, duration of presence in those zones, and time thresholds for processing unauthorized objects).

We used Python to train and run a deep learning model based on the YOLO neural network, which detects unauthorized objects and anomalous behavior. The next step was to test the solution on a real-time video stream, and if successful, implement it into the existing system.

Looking to add AI to your software? Our team can guide you from prototype to production.

Talk to Our Team

Challenges

Accurately identifying suspicious items left in specific zones for too long, without flagging harmless objects.
Defining and detecting unusual presence or activity in sensitive areas, such as prolonged loitering near restricted zones.
Seamlessly connecting Python-based YOLO AI detection with a .NET-based UI for real-time video monitoring and configuration.
Ensuring reliable 24/7 analysis using 100+ hours of video data, without delay or performance drops.
Automating alerts to reduce human reaction time.

Solutions

After analyzing the client’s current system, we took its specific features into account and began developing a roadmap for implementing the new solution.

Model Selection

Our first step was to select the most suitable pre-trained neural network model capable of detecting people and various objects. This was a key requirement, as the client did not plan to invest in training the model from scratch. At that time, we already had experience working with both commercial and free or freemium models. We chose YOLOv11, after which the client contacted the publisher to purchase a license for commercial use.

Data & Testing

For a preliminary evaluation on our side, the client provided data and 100+ hours of footage from the video surveillance cameras. These video materials were also used for further testing of the upcoming PoC application.

Development

For PoC development, our team chose C++, .NET, and Python as the main programming languages, which was also approved by the client’s technical team.

Detection Algorithms

To detect unauthorized objects and abnormal behavior, we developed algorithms that analyze the output from YOLO detection. This approach allows us to evaluate various object properties, such as type, size, and the duration of their presence in areas of interest. These areas are defined as specific sections of the image captured by either static surveillance cameras or cameras with dynamic viewing angles.

As a result, a suitcase or other object left within the camera’s field of view for a specified period of time is classified as an unauthorized object. The system then sends a notification to the operator for further review. Similar algorithms are used to analyze abnormal human behavior.

Integration

Our PoC application allows users to define zones within the video feed where YOLO detects people. If a person remains in such a zone longer than the specified duration, the system sends a notification via an external API to the operator for further analysis.

PoС testing was successful and approved by the client. Following that, we began implementing the solution into the existing system.

Outcomes

Successfully integrated the pre-trained YOLOv11 model to detect objects and people with high accuracy. The PoC passed testing and was approved by the client. The integration of the solution into the client’s security system was initiated.
With automated notifications, security teams can act within seconds of an incident.
As a result, operator response time was reduced by up to 80% – from an average of 5-7 minutes to 1 minute per incident – enabling the client’s security team to act before incidents escalate.
The client has continuous 24/7 surveillance analysis without operator fatigue.