Skip to content

[SPARK-17748][ML] Minor cleanups to one-pass linear regression with elastic net #15779

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
jkbradley wants to merge 3 commits into from
Closed

[SPARK-17748][ML] Minor cleanups to one-pass linear regression with elastic net #15779

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2bee341
Minor cleanups. Made SingularMatrixException private ml
jkbradley Nov 5, 2016
72a9302
expose max features for normal solver, and improve doc
jkbradley Nov 8, 2016
42ff98e
improve doc
jkbradley Nov 8, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 5 additions & 4 deletions mllib/src/main/scala/org/apache/spark/ml/optim/NormalEquationSolver.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,10 @@
*/
package org.apache.spark.ml.optim

import scala.collection.mutable

import breeze.linalg.{DenseVector => BDV}
import breeze.optimize.{CachedDiffFunction, DiffFunction, LBFGS => BreezeLBFGS, OWLQN => BreezeOWLQN}
import scala.collection.mutable

import org.apache.spark.ml.linalg.{BLAS, DenseVector, Vectors}
import org.apache.spark.mllib.linalg.CholeskyDecomposition
Expand Down Expand Up @@ -57,7 +58,7 @@ private[ml] sealed trait NormalEquationSolver {
*/
private[ml] class CholeskySolver extends NormalEquationSolver {

def solve(
override def solve(
bBar: Double,
bbBar: Double,
abBar: DenseVector,
Expand All @@ -80,7 +81,7 @@ private[ml] class QuasiNewtonSolver(
tol: Double,
l1RegFunc: Option[(Int) => Double]) extends NormalEquationSolver {

def solve(
override def solve(
bBar: Double,
bbBar: Double,
abBar: DenseVector,
Expand Down Expand Up @@ -156,7 +157,7 @@ private[ml] class QuasiNewtonSolver(
* Exception thrown when solving a linear system Ax = b for which the matrix A is non-invertible
* (singular).
*/
class SingularMatrixException(message: String, cause: Throwable)
private[ml] class SingularMatrixException(message: String, cause: Throwable)
Copy link
Contributor

@sethah sethah Nov 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

private[spark] as Jenkins was nice enough to point out

extends IllegalArgumentException(message, cause) {

def this(message: String) = this(message, null)
Expand Down
4 changes: 2 additions & 2 deletions mllib/src/main/scala/org/apache/spark/ml/optim/WeightedLeastSquares.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ private[ml] class WeightedLeastSquaresModel(
* formulation:
*
* min,,x,z,, 1/2 sum,,i,, w,,i,, (a,,i,,^T^ x + z - b,,i,,)^2^ / sum,,i,, w,,i,,
* + lambda / delta (1/2 (1 - alpha) sumj,, (sigma,,j,, x,,j,,)^2^
* + lambda / delta (1/2 (1 - alpha) sum,,j,, (sigma,,j,, x,,j,,)^2^
* + alpha sum,,j,, abs(sigma,,j,, x,,j,,)),
*
* where lambda is the regularization parameter, alpha is the ElasticNet mixing parameter,
Expand Down Expand Up @@ -91,7 +91,7 @@ private[ml] class WeightedLeastSquares(
require(elasticNetParam >= 0.0 && elasticNetParam <= 1.0,
s"elasticNetParam must be in [0, 1]: $elasticNetParam")
require(maxIter >= 0, s"maxIter must be a positive integer: $maxIter")
require(tol > 0, s"tol must be greater than zero: $tol")
require(tol >= 0.0, s"tol must be >= 0, but was set to $tol")

/**
* Creates a [[WeightedLeastSquaresModel]] from an RDD of [[Instance]]s.
Expand Down
5 changes: 4 additions & 1 deletion mllib/src/main/scala/org/apache/spark/ml/regression/LinearRegression.scala
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ import org.apache.spark.internal.Logging
import org.apache.spark.ml.feature.Instance
import org.apache.spark.ml.linalg.{Vector, Vectors}
import org.apache.spark.ml.linalg.BLAS._
import org.apache.spark.ml.optim.{NormalEquationSolver, WeightedLeastSquares}
import org.apache.spark.ml.optim.WeightedLeastSquares
import org.apache.spark.ml.PredictorParams
import org.apache.spark.ml.param.ParamMap
import org.apache.spark.ml.param.shared._
Expand Down Expand Up @@ -166,6 +166,9 @@ class LinearRegression @Since("1.3.0") (@Since("1.3.0") override val uid: String
* The default value is "auto" which means that the solver algorithm is
* selected automatically.
*
* The Normal Equation solver is limited to a few thousand features; when needed,
Copy link
Contributor

@sethah sethah Nov 5, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a bit misleading? If I set solver = "normal" and my dataset contains >4096 features, an exception is thrown instead of automatically falling back to iterative methods.

Copy link
Contributor

@yanboliang yanboliang Nov 7, 2016

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 @sethah It will only fall back when solver = "auto", we should clarify it more clear.

* [[LinearRegression]] will automatically fall back to iterative optimization methods.
*
* @group setParam
*/
@Since("1.6.0")
Expand Down