Assignment 13 - Where to start?

Master / Vertiefung Künstliche Intelligenz II
1

Disclaimer: Dieser Thread wurde aus dem alten Forum importiert. Daher werden eventuell nicht alle Formatierungen richtig angezeigt. Der ursprüngliche Thread beginnt im zweiten Post dieses Threads.

2

Assignment 13 - Where to start?
Hey everyone.

So I am not sure what to do in assignment 13. Are we supposed to do linear regression or linear classification? I know that we learned how to use the result for linear regression for linear classification but the learning rules change.

In case we are supposed to do linear regression:

  • Are we supposed to solve it analytically or via gradient descent or are we free to choose?

In case we are supposed to do linear classification:

  • When looking at slide 870 Definition 4.7.17 and below I tried to figure out what is going on here. Is the stuff below the definition just an explanation what will happen when applying the definition or do I actually have to make a case distinction here?

Appreciate any help. Kinda lost right now ^^

Best regards

3

Furthermore is this an error on the slides?

“Think of hw(x) =T(w·x)>0, whereT(z) = 0, if z≥0 and and T(x) = 0 otherwise. We call T a threshold function.”
Slide 870

Shouldnt it be T(z) = 1 if z > 0 and T(z) = 0 otherwise?

4

So I did implement it as a linear classification problem now but I don’t quite understand how it works. My problem is that I do not understand how the update rule on slide 870 works. The only part I do understand is that I do not have to update the weight if my classification is correct because the subtraction becomes zero. But how does the multiplication with x_i cause a correct update of the weight?

Another question: When applying this update rule (slide 870) in which order should it be done? Should I iterate the examples and update each weight or should I update a specific weight for all examples and then go to the next weight? Expressed as code:

for each example:
    for each weight:
        # update weight with single example

or

for each weight:
    for each example:
        # update weight with each example

best regards
nicfel

5

Well, the idea was to use linear regression to perform linear classification.
If you think about regression, you try to fit a line to your data which minimizes the distance to your samples.
If you now think about regression for classification, you have class labels which you try to predict. So you try to fit a line which minimizes the distance to the correct class label.

In general I would say you are free to choose, however the analytical solution is described in the slides so you could solve the model fitting in a single line of code :wink:

This information is not needed to solve the assignment
Well, you should distinguish between the type of classification and the algorithm you use. „Linear classification“ is a more general term and refers to „using a line as decision boundary“. Linear regression tries to do so by minimizing the distance to the correct class labels (regression task) but also logistic regression, SVM, perceptron and many more fit a line and are therefore linear classifiers as well.
So in short, the task was in this sence specific so you should use linear regression to perfrom linear classification.

This information is not needed to solve the assignment
Yes, it seems there are 2 typos in the slides.

This information is not needed to solve the assignment
The main information how to update your weights is described in the sign of the prediction error.

This information is not needed to solve the assignment
If you prefer gradient descent over the analytical solution, you should maybe take a look at the algorithm for multivariate linear regression (slide 866) which I suppose describes your problem. There you update every weight by summing over the classification differences for all samples.

I hope these answers help.

6

Thank you. I already submitted the code but the answers were interesting nonetheless.

best regards

nicfel