This is an introduction to a scripting engine that allows the C++ objects in the iLab Neuromorphic Vision Toolkit to be manipulated through a tcl scripting interface.
Table of contents:
For the impatient:
In saliency/ezvision.tcl is a translation of ezvision.C into script. The script version passes the ezvision test suite (do "make bin/invt; cd tests; ./test_scriptvision_blackbox.tcl") and the long movies test suite ("make bin/invt; cd tests; ./testlong_script_movies_blackbox.tcl"). It's less than 2x slower than the full C++ ezvision -- short ezvision test takes 13s with the C++ version vs. 23s with the tcl script (77% slower), while the long ezvision movies test takes 326s in C++ and 363s in Tcl (11% slower).
This system is based on some of the infrastructure from "GroovX" (http://ilab.usc.edu/rjpeters/groovx/). The imported source from groovx is in src/rutz (utility stuff in "rutz::" namespace), src/nub (object base class plus smart pointers in "nub::" namespace), and src/tcl (facilities for exposing c++ code through tcl scripts).
The only changes that are required to existing invt code are (1) ModelComponent derives from a new base class, nub::object, and (2) we use a new smart ptr type (nub::soft_ref) instead of SharedPtr to hold ModelComponent derivatives. More info below. In any case, the scripting layer is not intrusive in the sense that none of the core code (Image, Neuro, Devices, Media, etc.) has to know anything about scripting or scriptability, except that we have to use the right smart pointer types. The syntax for using the smart pointers themselves is identical between SharedPtr and nub::soft_ref.
The script interface for invt is defined in src/Script. The main program is in Script/invt.C -- it basically gives a list of the script modules that we want to load, and then calls off to some library in src/tcl to load those packages and then start looping, processing user input.
Then there are the script modules in src/Script. For now, we have:
For each C++ class that we want to export, we define a Classname_Init() function that sets up the script commands for that class. These *_Init() functions are what we enumerate in the main program. However, it is also possible to dynamically load shared libraries containing *_Init() functions into a running tcl program. That means that we could have true plug-ins, where new code never has to be linked into the main program, it just gets loaded at runtime.
Look at Script/NeuroScript.H. There we have Brain_Init() and Stdbrain_Init() (note that the functions are extern "C" and have the first letter capitalized, and all the rest lowercase -- this is important to allow dynamic loading -- that way if you tell tcl a package name, it can deterministically figure out what the name of the *_Init() function should be, and find it with dlsym()). Then there are also some functions for installing component types into the object factory, so that within a script we can create objects from their class name.
On to Script/NeuroScript.C.
Here's an overview of how the object-management system works. We use intrusive reference counting -- the ref count is within the object itself; we inject this into the ModelComponent hierarchy by having ModelComponent derive from nub::object. Because the reference count is intrusive, we don't have to worry about constructing multiple independent smart pointers from the same object, like we do with SharedPtr -- in this case, they always share the same reference count. We have nub::ref and nub::soft_ref smart pointer classes, nub::ref is "strong", meaning that its pointee can never be null. nub::soft_ref allows null pointees. Each nub::object gets a unique nub::uid when it is created. When we construct a nub::ref or a nub::soft_ref from an nub::object, we have the option to insert that nub::object into the nub::objectdb, which is essentially a std::map<nub::uid, soft_ref<nub::object>>. That is, it allows us to retrieve an object given just its nub::uid. This is how we expose objects to the tcl script -- via their nub::uid. So we are essentially using integer handles to the objects.
(BTW, I also have the ability to do weak pointers -- like weak references in Java -- that don't prevent their pointee from being destroyed. Instead, the weak pointers just silently become null when their pointee goes away -- so there's no risk of dangling pointers, and the weak pointers can be used to allow safe "back-pointers" up the object hierarchy without creating a strong-reference cycle that would prevent the entire cycle of objects from being destroyed.)
OK, skip down to Brain_Init() in Script/NeuroScript.C. We start a package with GVX_PKG_CREATE(). We inherit_pkg("ModelComponent") to bring in the script commands that were defined for ModelComponent (so we get "start" etc., see Modecomponent_Init() in Script/ModelScript.C to see what is exposed). We def_basic_type_cmds(), which allows us in script to find all current Brain objects, count them, query whether some object is a Brain or not. We registerComponentCreator<Brain>(), which puts a function into the nub::obj_factory (nub/objfactory.h) that knows how to create Brain objects. The creator function knows how to find a global ModelManager that is defined in ModelScript.C, and use that ModelManager to create the Brain.
Then we start getting into the pkg->def()'s. Here is where we are defining the script commands. The pkg->def() takes 4 params: (1) the name of the script command, (2) a human-readable string describing the arguments that the command takes, (3) the pointer-to-member or function address of the C++ function we want to make scriptable, and (4) SRC_POS, which is just a wrapper around __FILE__ and __LINE__ that lets us figure out, from inside a Tcl script, where a particular script wrapper was defined in the C++; that can be helpful for debugging.
What happens with pkg->def() is that we set up a big template thingy that gets the tcl script call as input (just a list of tcl objects), figures out how to convert those to native C++ types, then calls the C++ function, then converts the result back to Tcl. The command also does other nice stuff like ensure that the number of input arguments is correct, etc.
The c++ <-> tcl conversions are handled by a family of overloads that know how to do the individual conversions (basically like our convertToString() and convertFromString()). E.g., there is a converter from nub::ref->Tcl that returns the int value of the object's nub::uid, and a convert from Tcl->nub::ref that gets an int, then looks up an object with that nub::uid in the nub::objectdb, and casts it to the desired type.
Most of the time, the builtin c++ <-> tcl conversions will do the right thing, and it's possible to do a pkg->def() with just the address of an existing member or free function. Other times, it makes more sense to write a thin c++ wrapper, that does some more intelligent c++/tcl translation, that gets passed to pkg->def() instead. See for example brainEvolve() in Script/NeuroScript.C, which wraps Brain::evolve() so that instead of returning values through non-const reference parameters (which doesn't translate well into a script language with primarily value semantics), we just return all the return values in a tcl list, which we can unpack on the script side (see ezvision.tcl).
BTW, casting brings up the issue of exceptions -- each script command call is internally wrapped in a try/catch block (see tcl::command_group::invoke_raw in tcl/commandgroup.cc around line 382), so if any exceptions are raised while tcl is calling back to c++, the command machinery will catch the exception and translate into a human-readable error message on the script interpreter console (i.e., the program doesn't abort).
Here's an annotated session transcript. You can play along by doing "make bin/invt", then run "./bin/invt -nw" (the -nw means "nowindow"). The "./bin/invt 1>>>" is the prompt. You can use readline/history commands just like in bash (i.e., up-arrow to get previous commands, Esc-Del to kill the word behind the cursor, etc.). You can also use tab completion, but the only thing it will complete is filenames (i.e. it doesn't know how to complete script command names). Here goes:
[iLab9 19:51 7687]$ ./bin/invt -nw ########################################################################### ### iLab C++ Neuromorphic Vision Toolkit 3.1 (Jun 27 2005) ### ########################################################################### ### Copyright (c) 2001-2005 iLab and the Univ. of Southern California ### ### <http://ilab.usc.edu> ### ### Copyright (c) 1998-2004 Rob Peters and Caltech ### ### <http://ilab.usc.edu/rjpeters/groovx/> ### ### iLab C++ Neuromorphic Vision Toolkit is free software, covered by ### ### the GNU General Public License, and you are welcome to change it ### ### and/or distribute copies of it under certain conditions. ### ########################################################################### startup time (tcl+tk) 0.009s (user) 0.002s (sys) 0.010s (wall) startup time ( iNVT) 0.030s (user) 0.000s (sys) 0.030s (wall)
Let's find all current objects
./bin/invt 1>>> Obj::find_all
1
OK, there is one object, and its nub::uid is 1. Now let's see what its type is:
./bin/invt 2>>> Obj::type [Obj::find_all]
ModelManager
Cool, it's the global ModelManager that is constructed in Modelmanager_Init(). Let's see what params/subcomponents it currently has. We do "ModelManager::printout 1", where 1 is the nub::uid of the ModelManager.
./bin/invt 3>>> ModelManager::printout 1 model: ShowHelpMessage = true model: DebugMode = false model: UsingFPE = false model: TestMode = false model: TextLogFile = model: LoadConfigFile = model: SaveConfigFile =
In "real" code, we'd assign 1 to a variable with "set mm [ModelManager::find_all]", and then do "ModelManager::printout $mm". Let's do that now:
./bin/invt 4>>> set mm [ModelManager::find_all] 1 ./bin/invt 5>>> Obj::type $mm ModelManager
Now let's try to "printout" an invalid nub::uid:
./bin/invt 6>>> ModelManager::printout 3 rutz::error caught at src/tcl/commandgroup.cc:442: ModelManager::printout: at src/nub/refdetail.h:243: attempted to access invalid object in soft_ref<ModelComponent>
OK, it caught our invalid access attempt. The source file line numbers show where the exception was caught (src/tcl/commandgroup.cc:442) and where it was thrown (src/nub/refdetail.h:243)
Let's try to make a new object. First we try to create an object of type "blah".
./bin/invt 7>>> new blah rutz::error caught at src/tcl/tclcommandgroup.cc:442: new: at src/rutz/factory.h:224: known keys are: Brain InputFrameSeries OutputFrameSeries StdBrain unknown object type 'blah'
OK, it doesn't know anything about 'blah', but it tells us the types that it does know about. These are the ones that are in the nub::obj_factory, in which we registered them with registerComponentCreator().
Now let's make a brain, and capture its nub::uid in a variable called 'mybrain'
./bin/invt 8>>> set mybrain [new StdBrain] 2 ./bin/invt 9>>> puts "the value of mybrain is $mybrain" the value of mybrain is 2
OK, its nub::uid is 2. Now if we ask to find all Brain objects:
./bin/invt 9>>> StdBrain::find_all
2
./bin/invt 10>>> Brain::find_all
2
We get our StdBrain back from StdBrain::find_all. Note that Brain::find_all also finds our StdBrain even though StdBrain!=Brain, because StdBrain is a derived type of Brain.
./bin/invt 10>>> Obj::type $mybrain
StdBrain
There's its type. Now let's do Obj::find_all again:
./bin/invt 10>>> Obj::find_all
1 2 3 4 5 6 7 8 9 10 11 12 13
and get the object types:
./bin/invt 11>>> Obj::type [Obj::find_all]
ModelManager StdBrain RetinaConfigurator SaccadeControllerConfigurator
VisualCortexConfigurator SaliencyMapConfigurator
TaskRelevanceMapConfigurator AttentionGuidanceMapConfigurator
WinnerTakeAllConfigurator SimulationViewerConfigurator
InferoTemporalConfigurator GistEstimatorConfigurator ShapeEstimator
So there are all of the subcomponents of the StdBrain object that we just constructed.
Let's play around with our StdBrain some more. Query some of its param values:
./bin/invt 12>>> StdBrain::param $mybrain IORtype Auto
Now let's get a different param value -- and note that this is a good chance to make sure you're using the readline history editing feature of bin/invt: hit the up-arrow to retrieve the previous command (the one with 'IORtype'), then hit Esc-Delete to kill the 'IORtype' word, then type 'UseRandom' and hit Enter:
./bin/invt 13>>> StdBrain::param $mybrain UseRandom
true
Now, we can also change its UseRandom value
./bin/invt 14>>> StdBrain::param $mybrain UseRandom 0
1
(Note that the '1' returned here is the 'bool' value indicating whether a param named 'UseRandom' was found). Now let's re-query and see that our change took effect:
./bin/invt 15>>> StdBrain::param $mybrain UseRandom
false
OK, its value is now 'false'. Now let's try to give a bogus param value:
./bin/invt 16>>> StdBrain::param $mybrain UseRandom 3
StringConversions::convertFromString: Bogus format: 3 -- IGNORED
1
Let's make a "typo" and query a non-existent param:
./bin/invt 16>>> StdBrain::param $mybrain nosuchparam
[Brain]::getModelParamString: No parameter named 'nosuchparam' -- IGNORING
<unknown>
We can use the "?" function to get usage information about a command:
./bin/invt 18>>> ? Brain::param
Brain::param resolves to ::ModelComponent::param
::ModelComponent::param objref paramname (argc=[3..3])
::ModelComponent::param objref paramname paramvalue (argc=[4..4])
(defined at src/Script/ModelScript.C:118)
Here we see that Brain::param has two overloads, one that returns the current value, and one that sets a new value. The proper overload is selected at runtime based on the argument count.
We can apply "?" to itself:
./bin/invt 18>>> ? ?
? resolves to ::?
::? cmd_name (argc=[2..2])
(defined at src/tcl/tclpkg-misc.cc:156)
We can use "?" to see (1) which package originated the command, and (2) where it is defined. Here we see that Brain has inherited subCompByName from ModelComponent, and that it is defined in src/Script/ModelScript.C:
./bin/invt 19>>> ? Brain::subCompByName
Brain::subCompByName resolves to ::ModelComponent::subCompByName
::ModelComponent::subCompByName objref tagname (argc=[3..3])
(defined at src/Script/ModelScript.C:108)
We can query if an object is a (subclass of) a particular type:
./bin/invt 17>>> ModelComponent::is $mybrain
1
./bin/invt 18>>> StdBrain::is $mm
0
Try to get modules out of Brain:
./bin/invt 20>>> StdBrain::module $mybrain Retina 0 ./bin/invt 21>>> StdBrain::module $mybrain VisualCortex 0
OK, those returned null objects because the brain hasn't been start()ed yet.
Now let's try to get a invalid module type (note that this is an example of what happens with an LFATAL, which throws an exception, which is caught and reported to the script interpreter):
./bin/invt 22>>> StdBrain::module $mybrain Frobnaz
Brain::module: Invalid module type Frobnaz -- ABORT
-- ABORT.
exception of unknown type caught at src/tcl/commandgroup.cc:442:
StdBrain::module:
./bin/invt 23>>> StdBrain::hasModule $mybrain Frobnaz
Brain::hasModule: Invalid module type Frobnaz -- ABORT
-- ABORT.
exception of unknown type caught at src/tcl/commandgroup.cc:442:
StdBrain::hasModule:
Play around with our ModelManager some more:
./bin/invt 25>>> ModelManager::numSubComp $mm 0 ./bin/invt 26>>> ModelManager::tagName $mm model ./bin/invt 27>>> ModelManager::descriptiveName $mm iLab Neuromorphic Vision Toolit
Add our StdBrain to the ModelManager:
./bin/invt 28>>> ModelManager::addSubComponent $mm $mybrain 0
and now see how many subcomponents it has (you can use up-arrow to retrieve the identical call that we did 3 or 4 steps back):
./bin/invt 29>>> ModelManager::numSubComp $mm 1
Get the StdBrain back out of the ModelManager:
./bin/invt 31>>> ModelManager::subCompByIndex $mm 0
2
Try to get an out-of-bounds subcomponent:
./bin/invt 32>>> ModelManager::subCompByIndex $mm 123
[iLab Neuromorphic Vision Toolit]::subComponent:
iLab Neuromorphic Vision Toolit: request for
subcomponent 123 but I have only 1 -- FATAL
-- ABORT.
exception of unknown type caught at src/tcl/commandgroup.cc:442:
ModelManager::subCompByIndex:
See what's in the model manager now:
./bin/invt 33>>> ModelManager::printout $mm model: ShowHelpMessage = true model: DebugMode = false model: UsingFPE = false model: TestMode = false model: TextLogFile = model: LoadConfigFile = model: SaveConfigFile = model.Brain: IORtype = Auto model.Brain: UseRandom = false model.Brain: FOAradius = -1 model.Brain: FoveaRadius = -1 model.Brain: SimulationTimeStep = 0.0001 model.Brain: LevelSpec = 0-0,0-0,0 model.Brain: BrainBoringDelayInMs = 200 model.Brain: BrainBoringSMmv = 3 model.Brain: BrainMaxWinMv = 5 model.Brain: BlankBlink = false model.Brain: BrainSaveObjMask = false model.Brain: BrainTooManyShifts = 0 model.Brain: BrainTooMuchTime = 0 model.Brain: BrainSaveWinnerFeatures = false model.Brain.ShapeEstimator: ShapeEstimatorMode = FeatureMap model.Brain.ShapeEstimator: ShapeEstimatorSmoothMethod = Gaussian model.Brain.RetinaConfigurator: RetinaType = Std model.Brain.SaccadeControllerConfigurator: SaccadeControllerType = None model.Brain.VisualCortexConfigurator: VisualCortexType = None model.Brain.SaliencyMapConfigurator: SaliencyMapType = None model.Brain.TaskRelevanceMapConfigurator: TaskRelevanceMapType = None model.Brain.AttentionGuidanceMapConfigurator: AttentionGuidanceMapType = None model.Brain.WinnerTakeAllConfigurator: WinnerTakeAllType = None model.Brain.SimulationViewerConfigurator: SimulationViewerType = None model.Brain.InferoTemporalConfigurator: InferoTemporalType = None model.Brain.GistEstimatorConfigurator: GistEstimatorType = None
change the ModelManager's tag name:
./bin/invt 34>>> ModelManager::tagName $mm "someOtherName" ./bin/invt 35>>> ModelManager::tagName $mm someOtherName
Now, after all that, here's a shorthand syntax. For objects that have a tcl package, we can use "-> $obj cmdname" as shorthand for "<type>::cmdname $obj", where <type> is whatever is returned from "Obj::type $obj".
That is, "-> $mm printout " is equivalent to "ModelManager::printout $mm", because [Obj::type $mm] is "ModelManager":
./bin/invt 36>>> -> $mm tagName
someOtherName
./bin/invt 37>>> -> $mm tagName "someNewName"
someOtherName
./bin/invt 36>>> -> $mm tagName
someNewName
Finally, when we're done:
./bin/invt 39>>> exit
After we exit the interpreter, it leaves behind a prof.out file showing (1) average microseconds per call, (2) number of calls, (3) self microseconds, and (4) self+child microseconds, for each of the script commands (do "sort -n +2 prof.out | tail -20") to sort by total self microseconds. Here's one example left behind after running tests/test_scriptvision_blackbox.tcl, showing that Brain::evolve was called 1799 times, most of the time was spent in 6 calls to Brain::input, etc.
$ sort -n +2 prof.out | tail -20
291 1 291 291 tcl/ModelManager::extraArgs
123 3 369 369 tcl/ModelComponent::addSubComponent
242 2 484 484 tcl/FrameSeries::fileStem
246 2 492 492 tcl/Image::byte
1281 1 1281 1281 tcl/ModelManager::find_all
3843 8 1741 30745 tcl/::->
689 6 4139 4139 tcl/InputFrameSeries::readRGB
1955 3 5866 5866 tcl/Obj::new
5986 1 5986 5986 tcl/ModelComponent::start
1346 6 8080 8080 tcl/Brain::saveResults
6 1794 12075 12075 tcl/FrameSeries::isMultiframe
21874 1 21874 21874 tcl/ModelManager::parseCommandLine
6 3598 23255 23255 tcl/FrameSeries::didDisplayFrames
7 3599 27831 27831 tcl/FrameSeries::update
49 1799 88937 88937 tcl/Brain::evolve
162231 6 973388 973388 tcl/Brain::input
1.6.3