Activity and Context Recognition with Opportunistic Sensor Configurations


Specific outcomes include:

  • Spontaneous goal-oriented sensing ensembles
  • Robust abstract intermediate features
  • Novel opportunistic classifiers and opportunistic classifier fusion
  • Dynamic -short-term- adaptation of the activity recognition chain
  • Autonomous -long-term- evolution of the activity recognition system
  • Advances in Opportunistic robust Brain-Computer Interfaces

Application-wise, outcomes are activity and context recognition systems providing the following characteristics: (1) working over long periods of time despite changes in sensing infrastructure (sensor failures, degradation); (2) providing the freedom to users to change wearable device placement; (3) that can be deployed without user-specific training.


Anomaly detection in classifier ensemble [PRL]

In a multi-modal sensory recognition systems, sensors may provide anomalous behavior because of noise, failure, displacement, etc. Developing an anomaly detector for each sensor modality may not be feasible or it may need high effort. In specific cases where we use an ensemble of classifiers and each classifier is devoted to a subset of sensors and they are deciding among the same set of classes, we propose an approach to detect anomalous classifiers at the classifier fusion level. The pros and cons of these two levels of anomaly detection is provided in the table below.


  Modality-based Anomaly detection Fusion-level Anomaly detection

-Tuned for specific sensors (Fine view)

-Distributed computing

-Turn immediately the affected node off (Intelligent node)

-Recognize anomalous behavior in the network (global view)

-Independent of the sensor modality

-Independent of the type of anomaly

-Does not detect anomalies when the classifier is not affected

-Computation at the end (central node)


-Sometimes may not be feasible for a specific sensormodality

-Computations at node (Intelligent node)

-Dependency to the number of classes and classifiers

-Turning off the affected node by the central node

We proposed an information theoretic (IT) approach to detect anomalous classifiers. This is done by monitoring the mutual information between classifier decisions in a period of time.

We compared our approach with two other popular modality-level approaches, namely One Class SVM (OCSVM) and Generalized Log-likelihood Ratio (GLR).

In the figure below, we show the recognition performance before and after anomaly detection. In this case, we removed the anomalous classifiers from the ensemble and only fuse the decisions of other untouched classifiers. X-axis demonstrates how many classifiers (sensors) are affected by noise (rotation in a random direction and random degree). Here we have 5 accelerometer, 5 gyro and 5 magnetic sensors. The blue line shows the highest performance we can get by anomaly detection when it works perfectly. This is done in a supervised manner. The performance of IT method is almost equivalent to GLR method, however it is independent of the sensor and noise type.



Anomaly detection and adaptation [BSN13]

Upon detection of anomalous classifier, a retraining procedure starts. Since at run-time we don't have activity labels, we use the fusion decision (which is more reliable than any of the individual classifiers) to train the anomalous classifiers. The process is shown below:

For example, in the following we show a time-line of activities performed by the person.

In this example, we have eight sensors (and one classifier per sensor) and we artificially rotated four of the sensors (by rotating the sensor readings) with 60 degrees in a random direction.

The anomaly detection method was able to detect the rotated sensors [MJ Back,MJ RUA, MJ LUA, Knee]. However for some of them there is a latency.

Once an anomalous classifier is spotted we start retraining them. The figure below shows how the performance of the classifiers are evolved during retraining.

Finally, improvement of the individual classifier performance leads to the improvement of the fusion performance, as shown below. The upper bound is when the ground-truth labels are available at run-time, and the lower bound is when there is no adaptation process going on.


Do not hesitate to contact the consortium.


We develop opportunistic activity recognition systems: goal-oriented sensor assemblies spontaneously arise and self-organize to achieve a common activity and context recognition. We develop algorithms and architectures underlying context recognition in opportunistic systems.


Subscribe to our newsletter for regular project updates.

web design by
© 2018 OPPORTUNITY Consortium
The information on this website does not represent the opinion of the European Community, and is provided "as is". No guarantee or warranty is given that the information is fit for any particular purpose. The user uses the information at its sole risk and liability.