Interface for training and testing GistEstimatorBeyondBoF. More...
#include "Image/OpenCVUtil.H"#include <fstream>#include "Neuro/GistEstimatorBeyondBoF.H"#include "Neuro/StdBrain.H"#include "Neuro/NeuroOpts.H"#include "Neuro/NeuroSimEvents.H"#include "Media/SimFrameSeries.H"#include "Media/MediaOpts.H"#include "Simulation/SimEventQueue.H"#include "Simulation/SimEventQueueConfigurator.H"#include "Channels/ChannelOpts.H"#include "Component/ModelManager.H"#include "Component/ModelOptionDef.H"#include "Image/Point2D.H"#include "nub/ref.h"#include <torch/general.h>#include <torch/QCTrainer.h>#include <torch/SVMClassification.h>#include <torch/Kernel.h>#include <torch/MatDataSet.h>#include <glob.h>#include <unistd.h>#include <sstream>#include <ios>#include <numeric>#include <algorithm>#include <functional>#include <map>#include <vector>#include <iterator>#include <stdexcept>#include <utility>#include <limits>#include <cmath>
Go to the source code of this file.
Defines | |
| #define | BBOF_DEFAULT_TRAINING_DESCRIPTORS_FILE "sift_descriptors.txt" |
| #define | BBOF_DEFAULT_VOCABULARY_FILE "sift_vocabulary.txt" |
| #define | BBOF_DEFAULT_TRAINING_HISTOGRAMS_FILE "training_histograms.txt" |
| #define | BBOF_DEFAULT_SVM_CLASSIFIER_FILE "svm_classifier.txt" |
| #define | BBOF_DEFAULT_SVM_TEMP_FILE "/tmp/train-bbof-torch-dataset.txt" |
| #define | BBOF_DEFAULT_CLASSIFICATION_RESULTS_FILE "bbof_classifications.txt" |
| #define | BBOF_DEFAULT_IMAGE_NAME "some_image" |
| #define | BBOF_DEFAULT_SEGMENT_NUMBER "0" |
| #define | BBOF_DEFAULT_GIST_VECTORS_FILE "gist_vectors.txt" |
| #define | BBOF_DEFAULT_VOCABULARY_SIZE "200" |
| #define | BBOF_SIFT_CMD "sift" |
| #define | BBOF_VOCABULARY_CMD "vocab" |
| #define | BBOF_HISTOGRAM_CMD "hist" |
| #define | BBOF_SVM_CMD "svm" |
| #define | BBOF_CLASSIFY_CMD "classify" |
| #define | BBOF_CLASSIFY_GV_CMD "classify_gv" |
| #define | BBOF_ACTIONS |
| #define | BBOF_KMEANS_ITERATIONS (100) |
| #define | BBOF_KMEANS_PRECISION (.01) |
Functions | |
| template<typename T > | |
| static std::string | to_string (const T &t) |
| template<typename T > | |
| static T | from_string (const std::string &s, const T &defval=T()) |
| template<> | |
| std::string | from_string (const std::string &s, const std::string &) |
| int | main (int argc, const char *argv[]) |
Interface for training and testing GistEstimatorBeyondBoF.
The train-bbof program in conjunction with the GistEstimatorBeyondBoF class implements the following paper within the INVT framework:
Lazebnik, S., Schmid, C., Ponce, J. Beyond Bags of Features: Spatial Pyramid Matching for Recognizing Natural Scene Catgories CVPR, 2006.
Whereas the GistEstimatorBeyondBoF class is only concerned with the portions of the above paper that deal with gist vector computations, this program provides the remaining structure required to implement the necessary training and image classification functionalities.
train-bbof has two modes of operation, viz., training and testing. Training mode consists of four distinct phases: SIFT descriptor accumulation, K-means clustering, training histograms collection, and SVM generation. Testing mode operates in a single phase that uses the results of the clustering, histograms collection and SVM generation training phases to classify input images into appropriate categories.
Definition in file train-bbof.C.
| #define BBOF_ACTIONS |
("{"BBOF_SIFT_CMD"|"BBOF_VOCABULARY_CMD"|"\ BBOF_HISTOGRAM_CMD"|"BBOF_SVM_CMD"|"\ BBOF_CLASSIFY_CMD"|"BBOF_CLASSIFY_GV_CMD"}")
Definition at line 451 of file train-bbof.C.
| #define BBOF_DEFAULT_CLASSIFICATION_RESULTS_FILE "bbof_classifications.txt" |
In image classification mode, we write the results to a plain text file.
Definition at line 279 of file train-bbof.C.
| #define BBOF_DEFAULT_GIST_VECTORS_FILE "gist_vectors.txt" |
classification can be performed either with input images or with precomputed gist vectors. These precomputed vectors are stored in a plain text file. This is the default name of that file.
Definition at line 330 of file train-bbof.C.
| #define BBOF_DEFAULT_IMAGE_NAME "some_image" |
Several of the data files output by different operational modes of this program require inclusion of the current image/frame name and number and the ground truth segment/category number. These options allow users to specify appropriate values for this required info.
NOTE: The default values for these options are not very useful. They really ought to be explicitly specified on the command line.
Definition at line 301 of file train-bbof.C.
| #define BBOF_DEFAULT_SVM_CLASSIFIER_FILE "svm_classifier.txt" |
In the fourth phase of training, we create SVM classifiers for each of the categories and store the relevant parameters to a text file for later use during image classification. Each segment will have its own SVM classifier. Therefore, the default value of this symbol is not a good one to use and it should be explicitly specified on the command line.
Definition at line 246 of file train-bbof.C.
| #define BBOF_DEFAULT_SVM_TEMP_FILE "/tmp/train-bbof-torch-dataset.txt" |
While creating SVM classifiers for each of the categories, we need a temp file to store the training histograms data in the format required by the torch library. Usually, it would be a good idea to explicitly specify this on the command line rather than relying on the compiled in default.
Definition at line 264 of file train-bbof.C.
| #define BBOF_DEFAULT_TRAINING_DESCRIPTORS_FILE "sift_descriptors.txt" |
In the SIFT descriptors accumulation phase, we collect all the descriptors from the training images and store them in a plain text file.
Definition at line 186 of file train-bbof.C.
| #define BBOF_DEFAULT_TRAINING_HISTOGRAMS_FILE "training_histograms.txt" |
In the third phase of training, we compute and store the gist vectors for the training images. These gist vectors are used in the next training phase as the data points that will be used to create appropriate SVM classifiers for each image category.
Definition at line 223 of file train-bbof.C.
| #define BBOF_DEFAULT_VOCABULARY_FILE "sift_vocabulary.txt" |
In the second phase of training, we perform K-means clustering on the SIFT descriptors accumulated in the first phase and store the results in yet another plain text file.
Definition at line 203 of file train-bbof.C.
| #define BBOF_DEFAULT_VOCABULARY_SIZE "200" |
The vocabulary consists of a bunch of "prototypical" SIFT descriptors that are obtained by clustering the SIFT descriptors for the training images. It is possible to change the size of the vocabulary. But the default is 200.
Definition at line 352 of file train-bbof.C.
| #define BBOF_SIFT_CMD "sift" |
The different operational modes of this program must be specified as the one and only non-option command line argument. This "action" command must be one of the following strings (case-sensitive!):
1. sift -- accumulate the SIFT descriptors for the training images in the plain text file specified by the --sift-descriptors option. By default, the descriptors will be accumulated in ./sift_descriptors.txt.
Additionally, the --image-name and --segment-number options are required as this information is also recorded in the SIFT descriptors file.
2. vocab -- compute the SIFT descriptors vocabulary, i.e., the "protototypical" SIFT descriptors, from the accumulated descriptors using the OpenCV K-means implementation.
For this action, the --sift-descriptors option specifies the input file for the K-means while the --sift-vocabulary option specifies the output file. The defaults are to read from ./sift_descriptors.txt and write to ./sift_vocabulary.txt.
3. hist -- compute the flattened out multi-level histograms for the training set. The output is sent to the text file specified by the --histograms-file option.
The --image-name and --segment-number options are also required.
4. svm -- generate the SVM classifiers for each of the categories. The --svm-classifier file specifies the name of the file to which the SVM parameters will be stored. By default, this is ./svm_classifier.txt. Users should supply a file name different from the default. Otherwise, this file will get overwritten for each segment.
The --histograms-file can be used to specify the input data for this action.
In addition to the above two options, this action also needs the --svm-temp option to store the histograms data in the format required by the torch library. The default value is okay for this option. However, if several instances of this program can be executing in parallel, it would be best to supply different temp files explicitly on the command line.
5. classify -- uses the vocabulary and SVM classifiers produced by the vocab and svm actions to classify the input images streaming in. Classification results are written, by default, to ./classification_results.txt; but this can be changed with the --results-file option.
The --sift-vocabulary and --svm-classifier options can be used to specify appropriate values for the different pieces of input required by the classify action. Note that the --svm-classifier option does not point to a specific classifier, but really is a "root" name to use. This program will automatically load all the classifiers that begin with is this root. For example, if the user specifies --svm-classifier="ACB_svm_classifier", this program will load all the classifiers whose file names begin with "ACB_svm_classifier." and append numbers starting at 1.
6. classify_gv -- same action as classify except that input images are not used; instead, we assume that some other entity (e.g., a client application running on a phone) has computed the gist vector for one or more images and is passing these vectors via the --gist-vector option.
Definition at line 431 of file train-bbof.C.
| std::string from_string | ( | const std::string & | s, | |
| const std::string & | ||||
| ) | [inline] |
from_string() partial specialization for strings. If the client wants a string from the input string, we just return the input string. If we were apply the default version of this template function, we would end up parsing the input string as a whitespace separated string stream and only return the first string from this stream.
Definition at line 161 of file train-bbof.C.
| static T from_string | ( | const std::string & | s, | |
| const T & | defval = T() | |||
| ) | [inline, static] |
Read from string. As above, works as long as type T defines an operator >> that reads from an istream.
Definition at line 147 of file train-bbof.C.
1.6.3