Logistic Regression in Python – Limitations

As you can see from the examples above, applying logistic regression to machine learning is not difficult. However, it does have its limitations. Logistic regression cannot handle a large number of categorical features. In the examples we’ve discussed so far, we’ve reduced the number of features to a very large extent.

However, if these features are important in our predictions, we’re forced to include them, and then logistic regression won’t give us good accuracy. Logistic regression is also prone to overfitting. It can’t be applied to nonlinear problems. It performs poorly for independent variables that are unrelated to the target but correlate with each other. Therefore, you must carefully evaluate the suitability of logistic regression for your problem.

There are many other techniques designed specifically for the various areas of machine learning. To name a few, we have k-nearest neighbors (kNN), linear regression, support vector machines (SVM), decision trees, Naive Bayes, and many other algorithms. Before finalizing on a specific model, you must evaluate the suitability of these different techniques for the problem you are trying to solve.