NPTEL Introduction to Machine Learning Week 2 Assignment Answers 2024 (July-October)

As part of the NPTEL Introduction to Machine Learning course, students are tasked with a series of assignments designed to deepen their understanding of key concepts. In the Week 2 Assignment, participants encounter a range of questions that test their knowledge on regression methods, dimensionality reduction, encoding techniques, and more. Here, we provide detailed answers and explanations to assist students in their studies.

Question 1: True or False

• Typically, linear regression tends to underperform compared to k-nearest neighbor algorithms when dealing with high-dimensional input spaces.
  • Answer: True
  • Reason: Linear regression can suffer from overfitting in high-dimensional spaces where the number of features is much larger than the number of samples. K-nearest neighbors (k-NN), on the other hand, can perform better in such scenarios by making predictions based on local patterns in the data.

Question 2: Find the univariate regression function

• Given the following dataset, find the univariate regression function that best fits the dataset.

  x	y
  1	2
  2	3.5
  3	6.5
  4	9.5
  5	18.5

  • Answer: $y = 2x + 1$
  • Reason: By performing a linear regression on the given data points, we find that the best-fit line is $y = 2x + 1$. This can be determined through methods such as the least squares method, which minimizes the sum of the squares of the residuals between the observed and predicted values.

Question 3: Design matrix dimensions

• Given a training data set of 500 instances, with each input instance having 6 dimensions and each output being a scalar value, the dimensions of the design matrix used in applying linear regression to this data is:
  • Answer: $500 \times 7$
  • Reason: In linear regression, the design matrix includes a column of ones for the intercept term. Thus, with 6 input dimensions (features), the design matrix will have 7 columns (6 features + 1 intercept) and 500 rows (one for each instance).

Question 4: Assertion-Reason

• Assertion: A binary encoding is usually preferred over One-hot encoding to represent categorical data (e.g., colors, gender, etc.)
• Reason: Binary encoding is more memory efficient when compared to One-hot encoding.
  • Answer: Both A and R are true and R is the correct explanation of A
  • Reason: Binary encoding is more memory-efficient because it uses fewer columns to represent the same number of categories compared to One-hot encoding, which creates a separate column for each category.

Question 5: Select the TRUE statement

• Options:
  • Subset selection methods are more likely to improve test error by only focusing on the most important features and by reducing variance in the fit.
  • Subset selection methods are more likely to improve both bias and train error by focusing on the most important features and by reducing variance in the fit.
  • Subset selection methods don’t help in performance gain in any way.
  • Subset selection methods are more likely to improve test error and bias by focusing on the most important features and by reducing variance in the fit.
  • Subset selection methods don’t help in performance gain in any way.
  • Answer: Subset selection methods are more likely to improve test error by only focusing on the most important features and by reducing variance in the fit.
  • Reason: Subset selection methods like forward selection, backward elimination, and stepwise selection improve model performance by reducing overfitting, which can lead to lower test errors.

Question 6: Rank the subset selection methods in terms of computational efficiency

• Options:
  • Forward stepwise selection, best subset selection, and forward stepwise regression.
  • Forward stepwise selection, forward stepwise regression, and best subset selection.
  • Best subset selection, forward stepwise regression, and forward stepwise selection.
  • Best subset selection, forward stepwise selection, and forward stepwise regression.
  • Answer: Forward stepwise selection, forward stepwise regression, and best subset selection.
  • Reason: Forward stepwise selection is computationally less intensive than best subset selection, which requires fitting models for all possible subsets of predictors. Forward stepwise regression, which considers adding or removing predictors at each step, is more efficient than best subset selection but slightly less than forward stepwise selection.

Question 7: Choose the TRUE statements

• Options:
  • Ridge regression since it reduces the coefficients of all variables, makes the final fit a lot more interpretable.
  • Ridge regression since it doesn’t deal with a squared power of size to optimize than ridge regression.
  • Ridge regression has a more stable optimization than lasso regression.
  • Lasso regression is better suited for interpretability than ridge regression.
  • Answer: Lasso regression is better suited for interpretability than ridge regression.
  • Reason: Lasso regression performs feature selection by shrinking some coefficients to zero, leading to simpler and more interpretable models compared to ridge regression, which only shrinks coefficients but does not set any to zero.

Question 8: Which of the following statements are TRUE?

• Options:
• $\frac{1}{n} \sum_{i=1}^{n} a_i x_i, i = 1$
• $\frac{1}{n} \sum_{i=1}^{n} a_i x_{ij}, j = 1$
• Scaling at the start of performing PCA is done just for better numerical stability and computational benefits but plays no role in determining the final principal components of a dataset.
• The resultant vectors obtained when performing PCA on a dataset can vary based on the scale of the dataset.
• Answer:
  • Scaling at the start of performing PCA is done just for better numerical stability and computational benefits but plays no role in determining the final principal components of a dataset.
  • The resultant vectors obtained when performing PCA on a dataset can vary based on the scale of the dataset.
• Reason: Scaling the data before PCA ensures that each feature contributes equally to the analysis, which affects the resultant principal components. The principal components are sensitive to the scale of the data.

By tackling these questions, students enhance their understanding of critical machine learning concepts, preparing them for more advanced topics and practical applications. This assignment not only tests their theoretical knowledge but also hones their problem-solving skills, making it an integral part of the NPTEL Introduction to Machine Learning course.