C++ MLPACK :: LogisticRegression
- In the previous article in this series of MLpack with C++, we dealt with the algorithm named as LinearRegression.
- Here we will be seeing another important Machine-Learning algorithm named as LogisticRegression in MLpack.
Introduction :
- Logistic Regression is the regression analysis to conduct when the dependent variable is binary, i.e., can belong to one class or the other.
- Following are the two major questions (applications) that can be answered by Logistic Regression :
- How does the probability of getting lung cancer (yes vs. no) change for every additional pound a person is overweight and for every pack of cigarettes smoked per day?
- Do body weight, calorie intake, fat intake, and age have an influence on the probability of having a heart attack (yes vs. no)?
- The output value of Logistic-Regression-Function is of binary type (0 or 1), showing to which class, the given point belongs.
- Just like Linear regression assumes that the data follows a linear function, Logistic regression models the data using the sigmoid function.
- This sigmoid function is the one which takes all the output generated in the range [0, 1].
- Following is the chart-representation and mathematical formula for Sigmoid Function :
- In MLpack, LogisticRegression class implements an L2-regularized logistic regression model, and supports training with multiple optimizers and classification.
- The class supports different observation types via the MatType template parameter; for instance, logistic regression can be performed on sparse datasets by specifying arma::sp_mat as the MatType parameter.
- LogisticRegression can be used for general classification tasks, but the class is restricted to support only two classes. For multiclass logistic regression, MLpack provides a separate class called SoftmaxRegression (which we will see in some other article).
Model Description :
- The LogisticRegression is rather generally used as a classification technique as it outputs in form of binary variables.
- The model actually outputs in a continuous value, the same manner as the LinearRegression does, but using the Sigmoid Function, this continuous output is converted into discrete one.
Class Description :
- The class allows loading a logistic regression model (via the --input_model_file parameter) or training a logistic regression model, given training data (specified with the --training_file parameter), or both those things at once.
- In addition, this program allows classification on a test dataset (specified with the --test_file parameter) and the classification results may be saved with the --predictions_file output parameter.
- The trained logistic regression model may be saved using the --output_model_file output parameter.
- We can furthermore provide a lambda parameter (L2 regularization), which helps the model not to do over-fitting. This can be specified with the --lambda option.
- As an example, to train a logistic regression model on the data 'data.csv' with labels 'labels.csv' with L2 regularization of 0.1, saving the model to 'lr_model.bin', the following command may be used:
$ mlpack_logistic_regression --training_file data.csv --labels_file labels.csv --lambda 0.1 --output_model_file lr_model.bin
- Then, to use that model to predict classes for the dataset 'test.csv', storing the output predictions in 'predictions.csv', the following command may be used:
$ mlpack_logistic_regression --input_model_file lr_model.bin --test_file test.csv --output_file predictions.csv
Major Function Description :
- The class provides you with four constructors, the most widely used one is given below,
LogisticRegression ( const MatType & predictors, const arma::Row< size_t > & responses, const double lambda = 0)
The parameter description for the constructor is as follows :
predictors Input training variables. responses Outputs resulting from input training variables. lambda L2-regularization parameter.
- Next, comes the function to classify the provided new data-points (test dataset). Following is most commonly used template for this,
void Classify ( const MatType & dataset, arma::Row< size_t > & labels, const double decisionBoundary = 0.5) const
Parameters
dataset Set of points to classify. labels Predicted labels for each point. decisionBoundary Decision boundary (default 0.5).
- For calculating the accuracy of the model, we need to compare the actual outputs of data-points with the outputs that the model predicts. For this purpose, the class gives us a function called
double ComputeAccuracy ( const MatType & predictors, const arma::Row< size_t > & responses, const double decisionBoundary = 0.5)
Parameters
predictors Input predictors. responses Vector of responses. decisionBoundary Decision boundary (default 0.5).
- The accuracy is returned as a percentage, between 0 and 100. It returns the percentage of responses that are correctly predicted.
- The required header files are as follows :
#include <mlpack/core.hpp>
#include <mlpack/methods/logistic_regression/logistic_regression.hpp>
#include <ensmallen.hpp>
#include <boost/test/unit_test.hpp>
#include "test_tools.hpp"
using namespace mlpack;
using namespace mlpack::regression;
using namespace mlpack::distribution;
Stay connected for more Articles ...
Like the post if you find it worthy ...
Comments