Not working with regression problems #139
Comments
|
Any update on this issue? |
|
I'm also experiencing same issue. |
|
same issue |
|
same issue too |
|
Also same issue. So the package maybe didn't fit for regression? |
|
Hey, I had a similar problem and fixed this by passing |
Hi, how do you fix it? I used your method but still have error. |
|
Thats all I did, here is the code excerpt if it helps you: from keras.models import Sequential
from keras.layers.core import Dense, Dropout, Activation
from keras.layers import LeakyReLU
from keras.optimizers import SGD, Adam
from keras import optimizers
from elephas.ml_model import ElephasEstimator
model = Sequential()
model.add(Dense(128, input_dim=transformed_train_sample.head()['features'].size))
model.add(LeakyReLU(alpha=0.1))
model.add(Dense(64))
model.add(LeakyReLU(alpha=0.1))
model.add(Dense(32))
model.add(LeakyReLU(alpha=0.1))
model.add(Dense(16))
model.add(Activation('sigmoid'))
adam = optimizers.Adam(lr = 0.00001)
model.compile(loss='mean_squared_error', optimizer=adam)
sgd = optimizers.SGD(lr=0.01)
opt_conf = optimizers.serialize(sgd)
estimator = ElephasEstimator()
estimator.set_keras_model_config(model.to_yaml())
estimator.set_batch_size(32)
estimator.set_frequency('batch')
estimator.set_categorical_labels(True)
estimator.set_nb_classes(16)
estimator.setLabelCol("label_index")
estimator.setFeaturesCol("features")
estimator.setOutputCol("predictions")
estimator.set_loss('mean_squared_error')
estimator.set_optimizer_config(opt_conf)
estimator_pipeline = Pipeline(stages=[estimator])
estimator_model = estimator_pipeline.fit(transformed_train_sample)
#estimator_model.save("estimator_model")
eval_predictions = estimator_model.transform(transformed_eval_sample) |
Hi I think you are doing a classification work not regression |
|
As @leozzz620 commented what you have shown so far is not a regression. Changing the optimizer wouldn't be enough for the .fit to know you are working on something other than a classification problem. I'm trying to scour for some info on applying elephas on a regression problem, does the package even support it? The fit at the end should be applied to an x_train, y_train and validated on an x_test, y_test and that doesn't seem to be happening. Anyone have any progress? I'm not sure that Pyspark ml pipeline even supports regression. I've made it up to the fit model stage. Without this working, I'd have to undo all the work vectorizing the features in the first place and I'm not sure pandas can handle a dataframe of the magnitude i'm trying to work with. |
|
Sorry, I just commented since the error messages are really similar |
|
Support for regression was introduced in 0.4.5 - please retry and let me know if you run into any issues! |
|
It seems to be broken with new versions. I ran regression example on Google colab and gave these results: |
Hello and thank you for this package!
It seems that this package is not useful when the problem is regression rather than classification. After getting errors on my own data when using this package, I realized it always gives error for regression problems. As an example I slightly modified https://github.com/maxpumperla/elephas/blob/master/examples/ml_pipeline_otto.py to make it a regression problem from line 61 onwards:
Unfortunately it gives error:
The text was updated successfully, but these errors were encountered: