Machine Learning | Model Parameters
Modern predictive models typically do not make predictions out of the box. After choosing a model, it must be configured according to the training data it receives, and this process of configuration is called training or fitting the model. Fitting a model is similar to tuning a radio or an instrument—you alter some adjustable setting of the model until the output is the "best" in some mathematical sense, in the same way one might adjust the dial of a radio, or the pegs on a guitar until the sound output is optimal. These settings in a model are called the model's parameters, and adjusting a model's parameters to optimize the quality of its output is called tuning the parameters.
Example:
One relatively simple model is simple linear regression (also known as "least squares regression"), which tries to predict a variable y given another variable x by creating a line through the training data that follows the correlation between x and y present in the data and is the "best fit" in some mathematical sense (see [1] for more technical details).
Source: https://en.wikipedia.org/wiki/Linear_regression#/media/File:Linear_regression.svg
The graph shows the best fit line going through the data, which you can then use to make a prediction on y given any x. However, note that in order to generate this line, one would need two parameters—the slope and y-intercept for the line. Thus, in this example, training the model would mean tuning these parameters until the you get the line that best fits the data (again, see [1] for what "best fit" actually means on a technical level).
Limitations of Parameter Tuning:
It is important to note that parameter tuning is NOT a panacea that automatically solves problems with model choice or data quality.
Taking the tuning analogy again, for model choice, no matter how much you tune a radio on its FM setting, you will not get an AM station; tuning a guitar will also never make it able to produce the sounds of a piano. The modeler still has to pick the right model for the context of the data.
For data quality, too, there's a parallel here—tuning a radio will not help if the radio station itself is down, and a well-tuned guitar will not allow a poor guitarist to suddenly turn into a great one. The output of a model can only ever be as good as its input.
Footnotes/Resources:
[2] The typical parameter tuning process involves finding a minimum in the "loss function": Machine Learning | Loss Function