About

The eLog library was initially developed as a research prototype and later published for lifelogging researchers in 2010 to help them easily analyze heterogenous data and complex visualization. It has been kept growing with the progress of mobile computing environments and recently its UI part is released with GPL v3 license for wider usage. The eLog UI library is optimized for mobile environment and can get easily integrated with existing Web services.

Who We Are

The original work was proposed by Pil Ho and later extended the work with collaboration with 28 researchers around the world who contributed their lifelogs, collaborated for lifelog analysis and share research results to build up an open lifelogging platform for the public. Pil Ho has been keeping the development updating the library following up the progress in mobile computing.

Updates

  • Nov. 2014: Change the web page skin using bootstrap.
  • Nov. 2014: Published elog UI library as GPL v3.
  • Oct. 2014: Version up eLog library and documentation.

 

TLD optimization

TLD represents Z. Kalal, et. al's work (“Tracking-Learning-Detection.” IEEE transactions on pattern analysis and machine intelligence, vol. 34, Dec. 2011). Its open source is named OpenTLD in Matlab codes and they also have C++ version. This projects aims to port OpenTLD C++ version for Xcode (Mac) and speed optimization using OpenCV GPU supports.

Firstly, the original C++ version is successfully ported to Xcode (Mainly, manually creating Xcode projects and imports codes while fixing compatibility errors). Original codes are modified to run multiple objects at the same time. This is to build up unknow object models and to start creating lifelog image object model database. The issue is unlikely original Matlab version C++ version suffers performance issues. Internal code checking also arises concerns on now well optimized Matlab-to-OpenCV code conversion. This lab will record our efforts for speed optimization under Macbook, Xcode and GPU environments.

Main bottlenecks are tld::EnsembleClassifier::calcFernFeature, tld::IntegralImage:calcInitImg and tld:VarianceFilter:filter. Basically, the reason is because of the window size that linearly increases by the image size. 

numWindows += floor((float)(scanAreaW - w + ssw)/ssw)*floor((float)(scanAreaH - h + ssh) / ssh);

The above goes like like 191430 which means that it calls bottleneck functions (which is in fact a simple function) that many times. So the original author (who wrote this C++ version) tried to parallize this call by:

	
	#pragma omp parallel for
	for (int i = 0; i < numWindows; i++) {
        ...
        
		if(!varianceFilter->filter(i)) {
			detectionResult->posteriors[i] = 0;
			continue;
		}

		if(!ensembleClassifier->filter(i)) {
			continue;
		}

		if(!nnClassifier->filter(img, i)) {
			continue;
		}

So I turned on Xcode OpenMP support project option and the frame rate goes up about 80% faster.

See the above that omp_thread instances are initiated. I can also check from the CPU usage that all 4 CPU are working now. 

So we further applied openmp at many places (mostly for loops) around codes (See Main Thread percentage drops to 22.8%) but the FPS is not noticeably affected. 

Finally we stuided to find a way to include GPU acceleration in the process. Note that OpenMP is to use multiple CPU at the same time and GPU is in addition to use Graphic card CPUs. Anyhow its expected efforts are minimal since most computation times are spent in non-OpenCV GPU ported functions. We will revisit this issue later whether to port existing functions for GPU computation.

Additional topics we studied for TLD optimization are listed below: