Had my computer lying around...
Sunday, July 6, 2014
Been away from here for some time, basically I haven't been doing much fun stuff lately that seemed important to share here. But, last week, a friend asked me if I could help him counting people heads in a store. Basically, there is this very expensive hardware that use two cameras side by side, with an IR only filter and an IR emitter, so the idea is to find what is a head in a image and at what distance this head is from the camera pair, that's supposed to be installed on the ceiling, if the heads are in some range of height, they are probably human, so the camera counts plus one.
Ok so he asked me: can we do it? I thought for a minute, and replied "Yes, but you will need to buy some hardware to test it, don't know if it's worth the time", because it would take some time to receive the product and only then start to prototyping... And then it hit me! My laptop has stereoscopic camera! Emailed him, "give me two hours". Short story, in two hours, knowing near nothing on video processing, I could put together this code - using also code from around the web - that could do something near what he needed.
Ok, this codes needs to calculate the distance of something to the camera, so the idea is simple, get whatever moves more in one image, get the same for the other camera, assume that it's the same thing, and use math to calculate the distance. My equation came from this article: http://photon07.pd.infn.it:5210/users/dazzi/Thesis_doctorate/Info/Chapter_6/Stereoscopy_(Mrovlje).pdf
Viewing angle was measured using my bedroom wall, the points at each top corners of the screen where marked on the wall, and distance between then was measured and also the distance from the wall to the computer, the angle came from the resulting triangle. Did this once for each camera.
// First example on image processing
// Finds the square with bigger difference from the latter
// Do it for both cams
// Guess that it's in the same object
// Tell the distance from the object
// Prototype to be further rewritten in OpenCV
// This code uses snippets form this great german website:
// http://www.creativecoding.org/lesson/topics/video/video-in-processing
// Aktivität in Bildbereichen feststellen in Processing
import processing.video.*;
final int VIDEO_WIDTH = 320;
final int VIDEO_HEIGHT = 240;
final int VIDEO_COLS = 16;
final int VIDEO_ROWS = 12;
float[] activityR = new float[VIDEO_COLS * VIDEO_ROWS];
float[] buffer1R = new float[VIDEO_WIDTH * VIDEO_HEIGHT];
float[] buffer2R = new float[buffer1R.length];
float[] buffer3R = new float[buffer1R.length];
float[] activityL = new float[VIDEO_COLS * VIDEO_ROWS];
float[] buffer1L = new float[VIDEO_WIDTH * VIDEO_HEIGHT];
float[] buffer2L = new float[buffer1L.length];
float[] buffer3L = new float[buffer1L.length];
float Max=0;
int maxIndex;
Capture camR = null;
Capture camL = null;
int POSITIONXR = 0;
int POSITIONXL = 0;
float Distance = 0;
int maxIndexN(float[] array) {
float Max=0;
int maxIndex = 0;
for(int i = 0; i<array.length; i++){
if(array[i]> Max){
Max=array[i];
maxIndex = i;
}
}
return maxIndex;
}
void setup () {
size (640, 240);
camR = new Capture (this, VIDEO_WIDTH, VIDEO_HEIGHT,"LG 3D R Webcam", 30);
camL = new Capture (this, VIDEO_WIDTH, VIDEO_HEIGHT,"LG 3D L Webcam", 30);
frameRate (15);
camR.start();
camL.start();
}
void draw () {
if (camR.available ()) {
camR.read ();
int index;
int pxPerCol = VIDEO_WIDTH / VIDEO_COLS;
int pxPerRow = VIDEO_HEIGHT / VIDEO_ROWS;
image (camR, 0, 0);
for (int i=0; i < activityR.length; i++) {
activityR[i] = 0;
}
for (int i=0; i < camR.pixels.length; i++) {
//Calculates activity for the Right Camera
int x = (int) ((i % camR.width) / pxPerCol);
int y = (int) ((i / camR.width) / pxPerRow);
index = y * VIDEO_COLS + x;
color col = camR.pixels[i];
float sum = red (col) + green (col) + blue (col);
float deltaPixel = (buffer1R[i] + buffer2R[i] + buffer3R[i]) / 3 - sum;
if (deltaPixel < 0) {
deltaPixel *= -1;
}
activityR[index] += deltaPixel;
buffer3R[i] = buffer2R[i];
buffer2R[i] = buffer1R[i];
buffer1R[i] = sum;
}
int numeroQ = maxIndexN(activityR);
// This simply plots the X,Y position of the maxActivity index
textSize(32);
fill(255, 255, 255);
text(numeroQ%VIDEO_COLS, 10, 30);
text(",", 50, 30);
text(numeroQ/VIDEO_COLS, 60, 30);
POSITIONXR = numeroQ%VIDEO_COLS;
// Set activity for the right camera
for (int i=0; i < activityR.length; i++) {
activityR[i] /= pxPerCol* pxPerRow;
stroke (255, 20);
fill (0, 255, 230, activityR[i]);
rect ((i % VIDEO_COLS) * pxPerCol, (i / VIDEO_COLS) * pxPerRow, pxPerCol, pxPerRow);
}
}
if (camL.available ()) {
camL.read ();
int index;
int pxPerCol = VIDEO_WIDTH / VIDEO_COLS;
int pxPerRow = VIDEO_HEIGHT / VIDEO_ROWS;
image (camL, 320, 0);
for (int i=0; i < activityL.length; i++) {
activityL[i] = 0;
}
for (int i=0; i < camL.pixels.length; i++) {
// Calculate activity for the Left Camera
int x = (int) ((i % camL.width) / pxPerCol);
int y = (int) ((i / camL.width) / pxPerRow);
index = y * VIDEO_COLS + x;
color col = camL.pixels[i];
float sum = red (col) + green (col) + blue (col);
float deltaPixel = (buffer1L[i] + buffer2L[i] + buffer3L[i]) / 3 - sum;
if (deltaPixel < 0) {
deltaPixel *= -1;
}
activityL[index] += deltaPixel;
buffer3L[i] = buffer2L[i];
buffer2L[i] = buffer1L[i];
buffer1L[i] = sum;
}
int maxIndexN = maxIndexN(activityL);
// Just here to write the maxActivity X,Y position
textSize(32);
fill(255, 255, 255);
text(maxIndexN%VIDEO_COLS, 330, 30);
text(",", 370, 30);
text(maxIndexN/VIDEO_COLS, 380, 30);
POSITIONXL = maxIndexN%VIDEO_COLS;
for (int i=0; i < activityL.length; i++) {
//Just to ´rint out the activity
activityL[i] /= pxPerCol* pxPerRow;
stroke (255, 20);
fill (0, 255, 230, activityL[i]);
rect ((i % VIDEO_COLS) * pxPerCol+320, (i / VIDEO_COLS) * pxPerRow, pxPerCol, pxPerRow);
}
// Calculates distance, uses 50° as the viewing angle
Distance = 35*VIDEO_COLS / (2*tan(0.436)*(POSITIONXL-POSITIONXR));
// Prints out the calculated distance
textSize(32);
fill(255, 255, 255);
text(Distance, 200, 200);
text("DISTANCE =", 10, 200);
}
}
// Conclusion here is that the most hard is how to tell that whatever you select on left camera to find
// the distance from camera, what's the same thing on the right camera.
//
// Copyright 2014 Érico Porto
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.