So a micro-project is to generate HUDs for FPV Drone Racing… The coolest thing since sliced bread to uubergeeks

Truth is I honestly don’t know how i managed to get this working.. I had some pretty strange errors along the way but I’ve managed to do it so I wanted to note down how in case my laptop dies, is stolen or someone else wants to try.

What I want to do for phase 1:

  • Import a video file into Python (.mp4)
  • Convert to numpy array
  • Laplace filter it to detect edges.

Phase 1 was always going to be the hardest.

The first googling turned up this:

http://docs.scipy.org/doc/scipy/reference/tutorial/ndimage.html

 

That’s great but nor very user-friendly.

next search was this:

http://stackoverflow.com/questions/29718238/how-to-read-mp4-video-to-be-processed-by-scikit-image

More my level of skills (Thanks to the really bright people who can actually write libraries…), so this gives two options

  1. pylabs/imageio (imageio is the workhorse as pylabs was not referenced in the example)
  2. scikit-video

I’m using Anaconda for my python environment so i tried the usual
conda install imageio

But no luck, then I actually read the documentation (sorry!) and tried:
conda install -c pyzo imageio

That didn’t work, so i tried the PIP install method on the same page
pip install imageio

This should have done it but when I tried to restart the kernel in my ipython notebook and import it, it was not found in my packages…

Some faffing with my conda Environment (I had not set up an environment and was using root) gave more strange errors- even the example (conda create --name snowflakes biopython) would not work as a previously cocked up version of this command meant it kept looking for a package that did not exist (the name of my new environment videoedit)

Eventually after much faff I managed to make a new environment (not my exporting to txt as a lot of my packages were clashing (due to a recent numpy upgrade)
Richards-MBP:~ richardpeter$ conda create --name videoed --clone root
Source: /anaconda/anaconda
Destination: /anaconda/anaconda/envs/videoed
The following packages cannot be cloned out of the root environment:
- conda-4.1.6-py27_0
- conda-build-1.19.0-py27_0
Packages: 184
Files: 444
Linking packages ...
[ COMPLETE ]|###################################################| 100%
#
# To activate this environment, use:
# $ source activate videoed
#
# To deactivate this environment, use:
# $ source deactivate
#

Success! ?

Looking in my environment I could in fact see the imageio in there with <PIP> as the version… which meant it was in fact in my root so should have been in my jupyteer/ipython notebook all along! (!!!!?)

Anyway, I killed my current ipython session, did NOT enter my videoed environment and just ran the below in the terminal:

ipython notebook

It worked….  I don’t know why but it did.

 

2: Convert to numpy array:

After trying to edit one of my own mp4 files to a manageable 30s without success I downloaded this small, low res 1mb file from here.

A few lines of python later I’d flattened the image to a greyscale and plotted it:

 

Screen Shot 2016-07-10 at 20.51.54

 

3) Laplace Filter

Following the documentation  here a few more lines of python code give us a Laplace Filter:

from scipy.ndimage import correlate1d
from scipy.ndimage import generic_laplace

def d2(input, axis, output, mode, cval):
return correlate1d(input, [1, -2, 1], axis, output, mode, cval, 0)

a = np.zeros((3, 3))
a[1, 1] = 1
lf = generic_laplace(a, d2)

lf
array([[ 0., 1., 0.],
[ 1., -4., 1.],
[ 0., 1., 0.]])
Cool. And we can apply this to our image with a few more lines of python:

from scipy.ndimage import correlate
plt.imshow(correlate(img_grey, lf),cmap=’Greys_r’)

Screen Shot 2016-07-10 at 22.23.36

Somewhere in here you can vaguely see the outline of the scene.. but not quite what I was after… Perhaps a Sobel Filter instead? (And looking at the docs prewitt while we;re at it…)

from scipy.ndimage.filters import sobel, prewitt
plt.imshow(sobel(img_grey),cmap='Greys_r')

Screen Shot 2016-07-10 at 22.33.18

plt.imshow(prewitt(img_grey),cmap='Greys_r')

Screen Shot 2016-07-10 at 22.33.58

Hurrah! … But still not enough to be able to spot a competing drone…

Let’s try to make it more black/white… I wonder…

sob = sobel(img_grey)

sob.min(), sob.max()

(-560.66666666666663, 480.66666666666669)

Hmm…

To Dosob[sob>0] = sob[sob>0]*-1
plt.imshow(sob,cmap=’Greys_r’)

Screen Shot 2016-07-10 at 22.41.47

Better… but perhaps still not enough to identify a drone..

Googling edge detection in Python – looks like someone has been here before: https://en.wikipedia.org/wiki/Canny_edge_detector

And what I want is likely in openCV…

Tomorrow’s work…