Tom Goddard
11 AM, December 29, 2011
56 degrees Fahrenheit
Updated bee tracking report from 2015.
Traced paths of bees flying in and out of hive. Play video below with an HTML5 capable browser. Or view video track.mp4 (H.264) for Mac or Windows, or track.ogv (Theora) for Linux.
Original video taken with Sony TX7 digital camera using VGA mode (640x480). Low resolution used to speed up processing of bee flight paths. Video bees.mp4 (H.264) for Mac or Windows, or bees.ogv (Theora) for Linux. Or play image below with an HTML5 capable browser.
Average image frames of 60 seconds of video. Histogram of average image values (0-255) from Gimp. Peak around 220 is white portion of hive box. Peak around 170 is unpainted wood of hive box. Lower image intensities are from dark background. For identifying bees it worked better to compare images to a 2 second window of averaged images. This reduced false positives from wind induced noise (plants moving, camera moving) and shadows.
|
|
|
Variance of 60 seconds of video image frames. Brightness uses linear ramp peaking at value 4 using Gimp Curves tool. Histogram shows distribution of variance values. Bar heights are logarithmically scaled. Linear scaled heights shown at right.
| 
| 
|
Cumulative bee flights for durations from 0.5 to 60 seconds. Color coded by time, red = start, blue = end.
| 0.5 second | 
| 
| |
| 1 second | 
| 
| 
|
| 2 seconds | 
| 
| 
|
| 5 seconds | 
| 
| |
| 10 seconds | 
| 
| 
|
| 30 seconds | 
| 
| |
| 60 seconds | 
| 
| 
|
I wrote the software tracks.py that identifies and draws the bee flight paths one weekend. It is a primitive and experimental command-line program that requires Python (version 2.7 or newer, but will need fixes for Python 3), the Python Image Library, and the Numpy Python module. It only operates on images, and cannot read video formats. To convert video to a sequence of images I use ffmpeg. All of these packages run on Mac, Windows and Linux.
Here is a command example:
ffmpeg -i beevideo.mp4 image-%04d.png
python tracks.py image-%04d.png --start 1 --end 300 --animate
ffmpeg -i track-%04d.png beetracks.mp4
The first command turns the video into individual image frames, the second command draws bee flight lines, and the third command assembles the images with flight lines back into a video. The result beetracks.mp4 is a video with traced lines added for bee flight paths. An image file called tracks.png is also written that shows all the flight paths superposed on the background image.
Here is the help output from the tracks.py program.
./tracks.py --help
usage: tracks.py [-h] [--start i] [--end j] [--window n] [--smooth n] [--threshold level] [--spacing d] [--color {same,solid,time}]
[--nolines] [--animate] [--ave file] [--var file] [--out file] [--vidout file_pat] [--verbose n]
image_pattern
This program displays the flight paths of bees exiting and entering a hive by analyzing
a sequence of video image frames. It can combine the video frame images to make a single
images showing lines tracking each bee, and it can make a copy of the video frames with
lines added that trace the bee flight paths.
positional arguments:
image_pattern File name pattern for input images.
optional arguments:
-h, --help show this help message and exit
--start i First image frame file suffix number. Default 1.
--end j Last image frame file suffix number. Default 60.
--window n Number of frames in moving average for background image. Default 61.
--smooth n
Difference images (image minus average image) are smoothed where
each smoothing step sums adds to each pixel 1/4 times the four closest
neighbor pixels and divides by 2. Default 8.
--threshold level
Minimum color variation from background for detection.
Variation is sqrt(r*r + g*g + b*b) where r,g,b = color difference.
Color component range is usually 0-255. Default 10.0.
--spacing d
Minimum number of pixels between detected maxima in difference images.
A maximum position is removed if some higher maximum is closer
than this number of pixels. Default 12.0.
--color {same,solid,time}
How the color is chosen for coloring spots on the images where
variation is above the threshold. Choices are "same", "solid", or "time".
Solid colors high variation pixels red. Time colors
pixels by the frame number where highest variation
was seen proceeding from first to last frame using rainbow colors
red, orange, yellow, green, cyan, blue. Same means colors
the spot using the color from the image having largest variation from average.
This superimposes the bees from all frames in one image. Default same.
--nolines
Draw line tracks tracing bee paths on average image. Green lines are
used for accelerating motions, usually bees flying away from hive, otherwise red.
Bee locations for each frame are shown with yellow dots, with a yellow square
at the end of tracks where bees are in the final image frame.
--animate
Output each image frame with colored lines added for bee tracks.
The line shows the start of the bee track up to the position in the current frame.
The current end of a track (current bee location) is marked with a yellow square.
For frames after a bee track ends, the track line will be shortened
from where the track starts by N segments if the current frame is N frames beyond
the end frame of the track. This makes the tracks gradually disappear instead
of suddenly vanishing.
--ave file File to save average of images with no smoothing or windowing. Default None.
--var file File to save variation of images with no smoothing or windowing. Default None.
--out file File for image of all tracks superimposed on average image. Default tracks.png.
--vidout file_pat File pattern for output frames with bee track lines. Default track-%04d.png.
--verbose n Print diagnostic output for values 1 or larger. Default 0.
This program reads image files (e.g. png, jpeg, ppm formats) and cannot read video formats
directly. To convert video to individual images frames you can use the ffmpeg program
with a command like "ffmpeg -i beevideo.mp4 image-%04d.png". This program writes images.
To assemble a video with animated tracks use "ffmpeg -i track-%04d.png track.mp4"
This program requires Python (tested with version 2.7.5), and two Python libraries:
the Python Image Library (tested with version 1.1.7) for reading and writing images,
and numpy (tested with version 1.6.2) for doing the image analysis. The Python Image
Library is currently (2014) not available for Python 3. I tested on Mac OS 10.6 and 10.9.
The code should work on Linux and Windows as Python, PIL, numpy and ffmpeg are available
on all 3 operating systems.
The method for finding bee tracks is to compute an average image which represents the
static background of the video (hive box, plants, ...) then compare each video frame to
the average looking for differences which are flying bees. A bee is identified by looking
for pixel positions in frames that differ from the average background by more than a specified
threshold. The difference image is first smoothed to reduce false detections due to noise.
Then local maxima in the difference image are found. Weaker maxima are eliminated if they
are too close to another maxima. This avoids having one bee produce several maxima in a given
image frame. Then maxima in consecutive frames and close in space are joined to form a path.
The average image uses a moving window, typically consisting of 30 frames before and 30 frames
after the analyzed image frame or 1 second before and after for video at 30 frames / second.
A short time average is desirable to avoid changes in brightness due to moving clouds.
The method for tracking a bee from frame to frame is very basic. It starts at a bee position
in one frame and looks for the nearest bee position in the next frame. From that next frame
position it looks back to see if the nearest position in the current frame is the position
we started with, if not it rejects that connection. It also does not accept bee motions
from one frame to the next greater than 4 times the motion for that bee in the previous frame.
If a next frame position for a bee is not found it will look ahead one more frame, to account
for cases where a bee was not seen in a frame most likely because it was against a similar
color background. Paths are extended forward and backward in time and a bee position can
only be used in one path. If an extrapolated path goes outside the image frame, then the
nearest position in the next frame is not used and the path terminates. No momentum requirement
that prevents the bee from reversing direction too fast is imposed. There is currently no
code that rejects small plant motions -- so they are often seen as bee tracks when the wind
blows small leaves or flowers.
This code was tested with video at 640 x 480 pixels (small to speed up processing) with
the camera about 2 meters from the hive entrance. The camera is on a tripod. Hand-held video
is unlikely to work due to small camera motions. This code does not do any image stabilization.
Example output is shown at
http://sonic.net/~goddard/home/bees/bees.html
This program was just a weekend experiment, so its algorithm is primitive, and the implementation
is sloppy and slow. It will probably require fiddling with parameters such as the threshold and
spacing to work with videos with different resolution, lighting, or distance from bees.