TrackThemColors is an xtra for Macromedia Director that adds some rudimentary video scene analysis capabilities. (xtras are plug-ins that add functionality to Director.) It allows the creation of Director movies that incorporate motion, color, and pattern tracking. It also adds the ability to incorporate a live video feed as one of cast members. This is a step-by-step guide to using the xtra. Note that this is an attempt to present selected features of the TTC xtra. Some features are not illustrated. There is an annotated demo director movie available. Here are some links to add additional information
There are several stages in preparing an installation based on motion
detection that uses the TrackThemColors Xtra for Director.
Checking the video signal: Before you run any software you authored, you want to check whether there is a video signal. TrackThemColors will not run without a video signal. It is pretty good on connecting to video signals using the built in operating system functions. Run the trackthemcolors demo program first or any other program that reads the video signal (such as Premiere or Final Cut). If everything is working it will show a live feed of the current video signal. Then you are ready to go. (**Note if you use the TrackThemColors demo you must place the xtra as described below.)
Arrange the space: Postion the camera and lighting to minimize shadows and transient changes in lighting. Usually constant diffuse artificial lighting is the easiest to deal with. View the space through the video signal to determine what area is in view of the camera. Determine where the motion you are interested in shows up in the scene.
Place the Xtra: Add the xtra to the xtra folders in
the main Director folder. When Director starts up, it loads all the
xtras it can find in the special xtras folder. There is also
a way to explicitly tell Director where to find the Xtra.
Initialize the Xtra and start the video flow: Add these
routine to initialize the xtra and start the video flow. Add them
as scripts in the movie level. You should probably put
it in the on startmovie handler so it is automatically activated when Director
starts.
| --The initialization routines are put in a startMovie
handler
-- so they will be activated on startup -- the initialization routine is put in subroutine called setupTTC -- StartMovie
on StartMovie
end startMovie -- initvalues
on initvalues
-- setupTTC
on setupTTC
set TrackObj = new(temp)
-- startVideo
on startVideo
|
Cleanup - Closing the Xtra and stopping the video flow: TrackThemColors
wants you to release the xtra and video flow before you call those functions
again. It can cause memory problems and even malfunctions if this
is not done. The easiest way is to put them in an on stopMovie handler.
That way they will be automatically closed when the movie is stopped and
reactivated when it is started again.
| -- stopMovie
-- this routine closes the xtra by setting the global variable reference = 0 -- it also stops the video tracking by calling the cleanup() function -- the cleanup() function needs the variable trackObj reference so it should -- be called before the xtra is zeroed out -- if you call the new again before cleaning up you can crash the xtra or director -- the cleanup is only called when flag = 1 -- meaning video tracking has been inited on StopMovie
|
Pick the function you want to use - for example, motion, color,
brightness detection. Each one has a slightly different setup although
they are similar. This will illustrate the subroutine you could use
to detect motion. TrackThemColor's routines return rectangles that
indicate where it has found change (or the target color, etc.)
You must decide what to do with that information. For example, you
might see if the rectangles intersect known places in the scene (for example,
location of particular props) or see if the rectangles are moving to the
right or the left relative to where they were at the last reading.
The details of what you are interested in will be determined by your artistic
agenda. Director provides some built in functions to determine if
rectangles intersect, etc.
These routines assume that you have used the setup routines described
above in II to initialize the xtra and to start the video flow. The
reference to the xtra is put into a variable called trackobj.
One routine takes a snapshot of the reference frame (getsnap). The
other routine asks the xtra to report on any changes (checknew).
Your master routine desribed below (V) will call these routines and your
other routines (VI) will respond
| -- takeRef
-- SET UP REFERENCE SNAPSHOT -- this TakeRef routine sets up the reference frame -- it sets up a search rectangle to define the active video area -- it could be the whole rectangle of the video or some part -- for example, if you knew all the important action was on the right -- side of the screen, you would tell the xtra to ignore the left side -- this makes it slightly faster and cuts out sources of possible noise -- even if you do the whole frame, TTC documentation suggests -- taking off 10 pixels around the edge because video often has noise there -- it uses the vidrect global variable set in initvalues routine at start -- -- the remember() function is provided by the xtra to create the ref frame -- it takes 2 parameters - TrackObj which is the varuabke with the xtra -- set up in the initialization routine and -- a number that tells it where to get the video scence - 0 means live feed -- ohter numbers refer to cast members to use as refs -- -- NOTE: depending on the nature of your installation, it might be called -- once or several times. If the reference frame never changes (for example -- a video of the installation with no one in it, then you would call it once at -- the beginning) -- if there is something in the scene that is changing besides the motion -- of the people, then you would repeatedly call it to get a snapshot -- to compare against - for example a moving object or projected video or light change on takeRef
-- checkNew
on checkNew
GrabOneFrame(TrackObj)
-- get results
end checkNew |
Notice that the flag used in the idle is also used in the startMovie handler. If the flag is not 1 then none of the initialization routines will be called. This is useful if you are developing the parts of the movie that don't need active tracking. (see section II above)
The materVidalt routine shows how to do something different. Instead
of seeing if people have moved into target areas it just tracks motion
by putting a sprite at the place of their motion. A showVid routine
displays the current video feed on the stage when the showflag = 1.
| -- idle
-- USE IDLE TO CHECKTRACKING IN USE AND RESPOND -- Director constantly sends idle events -- this routine checks what interface it should use -- if flag = 1 then it goes to the video tracking using TTC -- that subroutine is arbitraily called 'mastervid' -- if flag = 2 then it goes to mouse based tracking -- that subroutine is arbitraily called 'mastermouse' -- if flag is any other value, then idle will do nothing -- this choice allows you to turn off video tracking while you are -- developing the program -- using the mouse to simulate movement -- you will need to type in "flag = 1or2or0" before you start to -- tell it what you want. Use the message box -- the variable showflag controls whether the video feed will be displayed -- if it is set to 1 it will call the showVid routine described below on Idle
if flag = 1 then masterVid
if flag = 2 then
end idle -- materVid
-- they use Director's intersect(rec1,rec2) function
on masterVid
-- see if in place #2-- go to appropriate
frame
-- branch to right section of movie
if newevent = 2 then
if newevent = 0 then
--put frect1&&frect2
--&& if you wanted to see what motion was detected
end if
-- masterVidalt
on masterVidalt
sprite(15).loc = z
end masterVidalt
-- showVid
on showVid
|
Here is the strategy. When the flag = 2 then the mouse interface
is active. 2 sprites become visible that represent the hot spots.
Moving the mouse pointer into those places is equivalent to someone walking
into a hot spot in the room.
| -- masterMouse
--this routine reads mouse movement as though it was person movement -- everything else in your movie will work the same if you set it up right -- the only thing different is the substitution for the intersect function -- this routine checks if the mouseloc is inside the hot rectangles -- Beware that the simulation is not perfect -- for example, ttc makes more errors than the mouse movement detection on masterMouse
-- see if in place #1 -- go to appropriate
frame
-- see if in place #2-- go to appropriate
frame
if curevent <> newevent then
if newevent = 2 then
if newevent = 0 then
end if end masterMouse |
This example assumes that there are 3 sections in the score. There are markers called event1,event2,event0. Each of these markers is the entry frame to a sequence that runs the events. At the end of these sequences there is an on exit handler that sends it back to the marker at the beginning. Each is a closed loop that can be exited only through movement of people or mouse equivalent. This example assumes that event0 is at frame 20; event1 is at frame40; and event2 is at frame60. It assumes that each sequence runs for 10 frames. These assumptions are completely arbitrary and would need to be customized with your artistic approach. It assumes that event0 is the default for when people have not moved to one of the hot spots.
You would need to set up the director movie by putting scripts in right
places and putting images,sounds,movies whereever. Just to show you
many options, this example makes an image become visible, starts a sound
playing, and jumps to a particular part of a digitial movie. There
are lots of ways in director to accomplish this or related media events.
| -- this script is put at beginning of event0
in frame script of frame 20
-- it assumes that sprite 10 has the appropriate image loaded for event0 -- it assumes that sprite 11 has event 1 image and sprite 12 have event 2 -- -- assumes that event0.wav is the appropriate sound for event 0 -- and that its link has been loaded into a cast member called sound0 -- it arbitraily plays it in sound channel 2 -- -- assumes that there is a link to a movie loaded into a cast member and t -- that movie is placed in sprite channel 13 -- place in script of frame 20
on enterFrame
sound(2).play(member("sound0")) sprite(13).movietime = 100
-- place in frame 30 script to loop within sequence
on exitFrame if soundbusy(2) = false then
go frame marker("event0")+1
|
| -- place in script of frame 40
-- turn off other images (channels 20 and 22); turn on event 1 (channel 21) -- start playing sound1 -- jump to position 400 of the movie in sprite channel 13 on enterFrame
sound(2).play(member("sound1")) sprite(13).movietime = 400
-- place in frame 50 script to loop within sequence
on exitFrame if soundbusy(2) = false then
go frame marker("event1")+1
|
| -- place in script of frame 60
-- turn off other images (channels 20 and 21); turn on event 2 (channel 22) -- start playing sound2 -- jump to position 600 of the movie in sprite channel 13 on enterFrame
sound(2).play(member("sound2")) sprite(13).movietime = 600
-- place in frame 70 script to loop within sequence
on exitFrame
go frame marker("event2")+1
|