The dataset we are using is one, but the dataset is divided into sub-datasets, each dataset consists of the same amount of data points as of the original dataset, each sub dataset is fed to a decision tree model, and as every sub-data is different from each other, we tend to get different results. To understand the above statement, let us dive into the architecture of gradient boosting-:ДЄbove is the architecture of a gradient boosting method, we can see that each decision tree is giving some prediction, but the question is that are they working on the same dataset? Because if they are working on the same dataset, they might be producing the same results which would mean there is no such benefit of these multiple predictions. If we place all the decision tree models in consecutive order, then we can say that each subsequent model will try to reduce the errors of the previous decision tree model. Gradient boosting integrates multiple machine learning models (mainly decision trees) and every decision tree model gives a prediction.

The question is which type of model does gradient boosting combine? This model generally combines several decision trees on sub-parts of the same dataset to form a stronger predictive model. (Recommended blog: What is Ensemble Learning?) Gradient boosting is a concept from ensemble learning, which by name, you can guess that it means assembling several models to form a strong model. Gradient Boosting is a method with which we try to increase the accuracy of our machine learning model, this method allows us to combine all the weak models, and after the combination of various weak models, we get a single model, which improves the accuracy of our model.