Wersja polska
Politechnika Gdańska CI TASK

Supercomputer Platform for Context Analysis of Data Streams in Identification of Specified Objects or Hazardous Events

Stream sources

Multimedia streams are used in a variety of domains, such as electronic surveillance of city streets, sports arenas or railway stations, as well as in medical examinations. KASKADA platform provides new solutions for effective management and analysis of this type of data.

Stream processing

KASKADA platform runs on Galera supercomputer in Academic Computer Centre TASK and allows for different kinds of multimedia stream analysis. Its innovative approach is based on parallel processing for multiple data inputs and providing real-time analysis results.

Analysis results

While the analysis is conducted, the processed parts of the streams are transmitted to external applications. When specified objects or events are detected, appropriate institutions can be notified through a notification system, e.g. email.

Development of services and applications

In the traditional development model, every author of a multimedia stream analysis application has at his disposal only some basic tools and libraries available in distributed processing systems (e.g. MPI). With these tools he has to single-handedly implement the whole set of necessary functionality, which takes a lot of effort.

The new model introduced in KASKADA platform allows for a significant reduction of the programmers' efforts through providing them with a whole spectrum of functions used in multimedia processing. Operations such as effective data transmission, support of popular multimedia codecs and monitoring of service execution are carried out automatically, which substantially reduces the complexity of the application's architecture.

The improved processing model additionally considers the possibility of using already existing services, stored in the KASKADA platform's repository. Creating new data analysis scenarios with ready-to-use, thoroughly tested components allows for a significant reduction of time necessary for deployment of new applications.

KASKADA Framework

  • set of libraries supporting algorithm implementation
  • efficient communication between tasks
  • automated multimedia stream processing

User Console

  • algorithms and services management
  • service execution monitoring
  • integration with external client applications
  • access through WWW and WebServices

Graphical Service Editor

  • graphical description language MSP‑ML
  • intuitive interface
  • fast deployment of new stream processing scenarios

Data repository

Stream archive

  • recording and storage of streams from various, distributed data sources
  • support for real-time preview and analysis

Test streams

  • testing of services with benchmark audio/video streams stored in the repository
  • archiving of multimedia streams uploaded by users

Storage server

  • disk array of 500 terabytes
  • distributed file system characterized by high availability and reliability

Service repository

Analysis services

  • a library of over 70 standard services for multimedia stream processing
  • service acceptance by experts guarantees high maturity and reliability
  • composition of new scenarios with existing services, through the graphical MSP-ML language
  • choosing the correct processing scenario basing on the context: place, time, weather, etc.

System services

  • multimedia stream decoding/encoding
  • monitoring of service execution and the platform environment
  • management of the service and data repositories

Service integration

  • conformity with standards such as WSDL, SOAP and UDDI
  • possibility to execute services directly from the user console
  • cooperation of services with external systems

Execution environment

Evironment management

  • effective mechanism for allocating tasks on the cluster
  • monitoring resource load of compute nodes

Efficient processing

  • distributed processing on multiple compute nodes
  • real-time analysis of hundreds of streams

Event handling

  • transmission of notifications about occurences of specified events
  • support for email and XMPP notifications

Software components

Java

  • Java: 92 934 lines of code
  • XML: 30 905 lines of code
  • JSP: 7 851 lines of code

C++

  • Framework: 22 054 lines of code
  • Algorithms: 9 425 lines of code
  • Monitor: 7 881 lines of code

User Console is used for creation, management and execution of services. Simple and complex services can be created by developers through a web interface. It is also responsible for management and supervision of the computing nodes and shows information about the state of individual nodes, received from the Monitor.

The Platform uses a task scheduler to assign tasks to computing nodes. This concerns both data processing services and tasks launched on the multimedia stream management server. The scheduler allows for effective utilisation of the available resources and increases reliability of the whole Platform.

The successful detection of specified situations causes generation of an event which is passed to the Event Management component. Each event is stored for administrative and safety purposes and could be delivered to the external recipients through e-mail or JMS queue.

Total: over 171 000 lines of code

Applications

Recognizing objects and events

The application performs real-time analysis of multimedia streams received by KASKADA platform from public places such as airports, railway stations, sports stadia or concert venues. The application's purpose is to identify specified objects and hazardous events, e.g.: potentially dangerous objects (e.g. firearms), suspicious unattended luggage, trespassing, brawls and riots, screams, shouts, calls for help, the sound of breaking glass, including car windows, gunshots, explosions, smoke. The application is also capable of interpreting crowd behaviour through audiovisual data analysis. The processing can be carried out in multiple threads without sacrificing accuracy of the results. The identified objects and events are stored in a repository. Through potential cooperation with agencies responsible for public safety, the application creates a possibility to improve the security of urban complexes and public facilities.

Supporting medical research

The application performs analysis of video streams from medical examinations performed with endoscopy devices and capsules. Although endoscopy makes it possible to quickly detect malignancies in the digestive system, examinations often take a lot of time and require full attention of a highly-skilled specialist before pathological changes are properly identified. The application can provide assistance by indicating the most important parts of the recording and suggesting the type of detected alterations in real-time. When the video is played again, only the important fragments with identified malignancies are presented, which reduces the amount of time needed to make the appropriate decisions. These activities can be also carried out remotely by using a supercomputer. The application enables KASKADA platform to cooperate with systems and devices used by clinics, hospitals and other medical facilities.

Intellectual property protection

The application analyses sets of documents (e.g. scientific works) and searches for any similarities in texts, mathematical formulae and illustrations. The comparative analysis uses as a base both a dedicated repository of digital documents, as well as the Internet and remote digital libraries. The platform allows for simultaneous analysis of multiple documents from a given category, which increases the accuracy and completeness of the results by providing a wider set of documents. The application can be used for detection of copyright infringement in scientific publications and other types of reports, documents and software, as well as for improving methods of searching for specific data available in various repositories.

Creating virtual portraits

Virtual Portrait application allows for creating hypothetical liknesses of people whose actual appearance remains unknown. By using existing historical data, our algorithm generates propositions of believable portraits, potentially resembling the person we are looking for.
The utilized algorithm belongs to the family of so-called genetic algorithms (GAs). The general idea is to search for the optimal solution through a process resembling biological evolution. During the algorithm execution, many potential solutions are generated, to which the algorithm cyclically applies procedures of cross-over (combining features of different solutions) and mutation (introducing small random fluctuations to the solutions).
The application was used for creating a virtual portrait of the world-famous Gdańsk scientist, Daniel Gabriel Fahrenheit.

Business integration

Functionality implemented in KASKADA platform allows for broad integration with business and public units. Through negotiating appropriate contracts by intermediate agencies, applications can be used by emergency services such as medical emergency or fire services.

Quality analysis

One of the basic aims of KASKADA platform development was to assure high quality of provided services. Through utilising the computing power of the Galera supercomputer, as well as using advanced resource allocation algorithms and efficient data stream processing techniques, high scalability of the platform was ensured, which allows for execution of a large number of simultaneously working services. Additionally, specific features of parallel real-time data processing were taken into consideration, so that the services running in KASKADA platform are characterised by low data loss, which corresponds to high quality of the obtained results.

Our team

MAYDAY EURO 2012 project coordinator

Prof. Henryk Krawczyk, PhD Eng

Prof. Henryk Krawczyk is a specialist in distributed architecture computer systems and software engineering. He is the head of the Department of Computer Architecture and was the dean of Faculty of Electronics, Telecommunications and Informatics for four times. Since 2008 he has been the rector of Gdańsk University of Technology. He promoted 22 PhD theses. He participated in many successful national and international research projects (e.g. Tempus, Copernicus, Esprit and Mayday Euro 2012 – one of the most innovative research grants) and is a co-author of two licences (2010, 2011) in a research project linked with VoIP systems. He is an author of several computer system implementations (e.g. a support system for endoscopy examination used by Medical University of Gdańsk) and more than 350 scientific publications (with 99 external citations, Index-H: 3), including one foreign monograph and nine KASKBOOK books related to research works of the Department of Computer Architecture. He is a member of the Polish Academy of Sciences, chairman of the CI TASK Users Council and WETI, IBS PAN and IPI PAN Scientific Committees. He is a member of the KRASP Praesidium and president of the Committee on Information Infrastructure of KRASP.

KASKADA team is a close-knit group of nine passionate individuals from Gdańsk University of Technology who are enthusiastic about high performance computing and multimedia processing. In our team we have one PhD., six MSc. Eng. (including five PhD. students) and two Eng. We specialize in C++ (e.g. RDMA, MPI, Boost and GStreamer) and Java technologies such as Spring, Hibernate and ActiveMQ. Our developers have rich experience in scientific and business projects, such as Beesy Cluster, Tempus, Copernicus or Endoscopy Recommender System (ERS). Part of our team have certifications from IBM, Oracle/Sun, Novell and LPI (e.g. OCPJP and LPIC). We focus not only on development, but we also do research that is connected with HPC. We create models based on diagnostic data, we design and execute validation tests to prove them, and we use them to solve real-life problems. We have published more than 15 publications in national and international journals and conferences.

  • Barłomiej Daca

  • Adam Glinianowicz

  • Rafał Knopa

  • Michał Nykiel

  • Piotr Orzechowski

  • Piotr Sawicki

  • Karol Zalewski

  • Tomasz Ziółkowski

Team Leader

Jerzy Proficz, PhD Eng

Jerzy is responsible for tough decisions that influence KASKADA team's aims and work style. He has rich experience in creating multimedia platforms using SOA architecture, high performance computing and fast fiber networking. He was responsible for designing and implementing the system framework for pilot applications related to automated video security, medical consultations and copyright infringement detection. He received his PhD. in parallel and distributed processing in 2012 from Gdańsk University of Technology. His research interests include parallel and distributed architectures and multimedia processing. Jerzy is an IBM Certified Enterprise Developer and Sun Certified Enterprise Architect.

Publications

During the course of the project, our team have authored a number of articles in scientific journals and international conferences. These publications concern subjects such as parallel processing, task scheduling and multimedia services modeling. Members of KASKADA team have also written five master's theses (one still in progress) and one PhD dissertation. Below, we present a list of the most important papers.

Articles

  • Krawczyk H., Proficz J. Real-time multimedia stream data processing in a supercomputer environment, Interactive Multimedia, InTech, 2012 [PDF]
  • Bobkowska A., Nykiel M., Proficz J. Evaluation of Multimedia Stream Processing Modeling Language from the Perspective of Cognitive Dimensions, Proceedings of PPIG 2011, York, 2011
  • Krawczyk H., Proficz J., Bańczyk K. Parallel processing of multimedia streams, Transactions on Computer Applications in Electrical Engineering, Poznan, April 19-21, 2010, pp. 267-268
  • Krawczyk H., Proficz J. KASKADA – multimedia processing platform architecture, Proceedings of the International Conference on Signal Processing and Multimedia Applications, SIGMAP 2010
  • Krawczyk H., Proficz J. The task graph assignment for KASKADA platform, Proceedings of the International Conference on Software and Data Technologies, ICSOFT 2010
  • Krawczyk H., Proficz J., Bańczyk K. Parallel processing of multimedia streams, Computer Applications in Electrical Engineering, vol. 8 (2010)
  • Krawczyk H., Konpa R., Proficz J. Basic management strategies on KASKADA platform, EUROCON 2011
  • Krawczyk H., Proficz J., Ziółkowski T., A Parallel Genetic Algorithm for Creating Virtual Portraits of Historical Figures, TASK Quarterly 2012

PhD thesis

  • Proficz J. Management of computation resources for multimedia stream processing in a cluster environment, 2012

Master's theses

  • Daca B. Multimedia communication protocols for KASKADA platform (in progress)
  • Glinianowicz A. The KASKADA platform – mechanisms of computation resources allocation, 2012
  • Nykiel M. Complex services development and validation methods for the KASKADA platform, 2011
  • Orzechowski P. Monitoring and fault tolerance of the KASKADA platform's web services, 2011
  • Ziółkowski T. Management and processing of data streams in the KASKADA platform, 2011

Contact information

Address:

Gdańsk University of Technology,
TASK Academic Computer Centre,
G. Narutowicza 11/12
80-233 Gdańsk

Phone:

+48 58 348 63 43