Comments on This Number Crunching Life: Python logistic regression (with L2 regularization...

Got it! Thanks.... That'll teach me to be more...

2011-12-09T15:09:21.886-08:00

Got it! Thanks.... That'll teach me to be more careful when reading long eqns...

Hi Steven, the (k>0) should only zero out the c...

2011-12-09T12:38:38.941-08:00

Hi Steven, the (k>0) should only zero out the contribution to the gradient of the regularization term. So it's saying that we don't care a priori about keeping the bias term small. The full gradient for the bias term should not in general be 0.

This is a silly question, but in your dB_k functio...

2011-12-09T12:05:03.289-08:00

This is a silly question, but in your dB_k function, everything is multiplied by (k>0). Since the bias term (I think) corresponds to the k=0 term of the beta vector, it looks like you're keeping the bias term fixed at 0, since (k>0) = 0 when k = 0.

Am I misunderstanding something here, or was this a purposeful choice?

This is useful, thanks. Currently this works for ...

2011-12-05T16:15:17.378-08:00

This is useful, thanks.

Currently this works for a single dependent variable/y. Do you think it could be extended to multiple y e.g. for categorization purposes?

A wonderful Post

2011-07-22T09:49:08.072-07:00

A wonderful Post

2011-07-22T09:48:58.323-07:00

A wonderful Post

What is the license on this code? Do you allow co...

2011-06-07T10:40:35.739-07:00

What is the license on this code? Do you allow commercial use?

A wonderful Post! Thanks a lot! Also thanks to Ja...

2010-11-01T16:20:10.445-07:00

A wonderful Post! Thanks a lot!
Also thanks to Jason for his tips.

Thanks, Jason. I appreciate the (very good) tips....

2009-10-15T07:55:53.165-07:00

Thanks, Jason. I appreciate the (very good) tips.

When I get some spare time, maybe I'll walk down the hall to George's office and hammer out a really clean version of this.

A few tips: * The basic approach of using classes...

2009-10-15T06:23:31.694-07:00

A few tips:

* The basic approach of using classes is a good one---avoids you needing to pass arguments via the optimizer to objective/gradient calculations.

* Test data, the parameter vector and the optimization method should not be part of the class or instance. You should be able to interchange these w/o changing your model.

* Your class should define (1

Alex, Do you mean my other post on march madness predictions? It was fairly popular on Digg and a few small news outlets, but it would be news to me if it made it to ESPN.

didn't i see this exact example on ESPN just b...

2009-08-04T15:08:40.775-07:00

didn't i see this exact example on ESPN just before MM?

So first off, I'm far from being a great softw...

2009-06-07T13:04:42.904-07:00

So first off, I'm far from being a great software architect, and I really haven't thought about object oriented design since second year of undergrad, so I don't have super strong feelings about this.

But I'll still bite. I probably buy the argument with respect to the test data -- it may be cleaner not to store it as an instance variable, since it's not really a &

I guess the use of the class keyword isn't rea...

2009-06-06T22:28:24.843-07:00

I guess the use of the class keyword isn't really what bothers me, it is that the class has the training and testing data as instance data. Using class or not is a mostly irrelevant implementation detail. What happens if a user of your class allows the data instance variables to be their defaults of None and then tries to call a function? I expect an ungraceful crash. I am not suggesting

Good point about the single predict function -- I&...

2009-06-06T13:34:54.644-07:00

Good point about the single predict function -- I'll probably change mine to work the same way. I also haven't used the getopt module, but that sounds useful. Thanks for the pointer.

As for the classes, though, I'm pretty deep in the make-everything-a-class boat, so we'll have to agree to disagree on that =).

Not bad. I have my own simple python 2-class logi...

2009-06-06T09:32:31.875-07:00

Not bad. I have my own simple python 2-class logistic regression implementation (supports L1 and L2 regularization) that only depends on numpy (I try to avoid dependencies on the rest of scipy and matplotlib which means the calling code has to plot things).

A few differences. I a slightly tweaked version of Roland's python port of Carl Rasmussen's minimize.m instead of the