logo
down
shadow

sklearn use RandomizedSearchCV with custom metrics and catch Exceptions


sklearn use RandomizedSearchCV with custom metrics and catch Exceptions

By : user2171899
Date : October 25 2020, 09:10 AM
hop of those help? You can have a generic scorer which can take other scorers as input, check the results, catch any exceptions they throw and return a fixed value on them.
code :
def custom_scorer(y_true, y_pred, actual_scorer):
    score = np.nan

    try:
      score = actual_scorer(y_true, y_pred)
    except Exception: 
      pass

    return score
acc = make_scorer(custom_scorer, actual_scorer = accuracy_score)
auc_score = make_scorer(custom_scorer, actual_scorer = roc_auc_score, 
                        needs_threshold=True) # <== Added this to get correct roc
recall = make_scorer(custom_scorer, actual_scorer = recall_score)
mcc = make_scorer(custom_scorer, actual_scorer = matthews_corrcoef)
bal_acc = make_scorer(custom_scorer, actual_scorer = balanced_accuracy_score)
import numpy as np
def custom_scorer(y_true, y_pred, actual_scorer):
    score = np.nan

    try:
      score = actual_scorer(y_true, y_pred)
    except Exception: 
      pass

    return score


from sklearn.metrics import make_scorer, roc_auc_score, accuracy_score
acc = make_scorer(custom_scorer, actual_scorer = accuracy_score)
auc_score = make_scorer(custom_scorer, actual_scorer = roc_auc_score, 
                        needs_threshold=True) # <== Added this to get correct roc

from sklearn.datasets import load_iris
X, y = load_iris().data, load_iris().target

from sklearn.tree import DecisionTreeClassifier
from sklearn.model_selection import GridSearchCV, KFold
cvv = KFold(3)
params={'criterion':['gini', 'entropy']}
gc = GridSearchCV(DecisionTreeClassifier(), param_grid=params, cv =cvv, 
                  scoring={"roc_auc": auc_score, "accuracy": acc}, 
                  refit="roc_auc", n_jobs=-1, 
                  return_train_score = True, iid=False)
gc.fit(X, y)
print(gc.cv_results_)


Share : facebook icon twitter icon
sklearn: use Pipeline in a RandomizedSearchCV?

sklearn: use Pipeline in a RandomizedSearchCV?


By : user3088360
Date : March 29 2020, 07:55 AM
seems to work fine RandomizedSearchCV, as well as GridSearchCV, do support pipelines (in fact, they're independent of their implementation, and pipelines are designed to be equivalent to usual classifiers).
The key to the issue is pretty straightforward if you think, what parameters should search be done over. Since pipeline consists of many objects (several transformers + a classifier), one may want to find optimal parameters both for the classifier and transformers. Thus, you need to somehow distinguish where to get / set properties from / to.
code :
param_dist = {
          'rbf_svm__C': [1, 10, 100, 1000], 
          'rbf_svm__gamma': [0.001, 0.0001], 
          'rbf_svm__kernel': ['rbf', 'linear'],
}
How would you do RandomizedSearchCV with VotingClassifier for Sklearn?

How would you do RandomizedSearchCV with VotingClassifier for Sklearn?


By : alfredhichcock
Date : March 29 2020, 07:55 AM
To fix the issue you can do You can perfectly combine both, the VotingClassifier with RandomizedSearchCV. No need to run them separately. See the documentation: http://scikit-learn.org/stable/modules/ensemble.html#using-the-votingclassifier-with-gridsearch
The trick is to prefix your params list with your estimator name. For example, if you have created a RandomForest estimator and you created it as ('rf',clf2) then you can set up its parameters in the form . Specific example: rf__n_estimators: [20,200], so you refer to a specific estimator and set values to test for a specific param.
code :
import numpy as np
from sklearn.ensemble import RandomForestClassifier
from sklearn.ensemble import VotingClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.grid_search import RandomizedSearchCV

X = np.array([[-1, -1], [-2, -1], [-3, -2], [1, 1], [2, 1], [3, 2]])
y = np.array([1, 1, 1, 2, 2, 2])

clf1 = DecisionTreeClassifier()
clf2 = RandomForestClassifier(random_state=1)

params = {'dt__max_depth': [5, 10], 'rf__n_estimators': [20, 200],}


eclf = VotingClassifier(estimators=[('dt', clf1), ('rf', clf2)], voting='hard')

random_search = RandomizedSearchCV(eclf, param_distributions=params,n_iter=4)
random_search.fit(X, y)
print(random_search.grid_scores_)
sklearn custom scorer multiple metrics at once

sklearn custom scorer multiple metrics at once


By : Suba
Date : November 20 2020, 11:01 PM
will help you In short: you cannot.
Long version: scorer has to return a single scalar, since it is something that can be used for model selection, and in general - comparing objects. Since there is no such thing as a complete ordering over vector spaces - you cannot return a vector inside a scorer (or dictionary, but from mathematical perspective it might be seen as a vector). Furthermore, even other use cases, like doing cross validation does not support arbitrary structured objects as a return value since they try to call np.mean over the list of the values, and this operation is not defined for the list of python dictionaries (which your method returns).
sklearn RandomizedSearchCV with Pipelined KerasClassifier

sklearn RandomizedSearchCV with Pipelined KerasClassifier


By : prem singh
Date : March 29 2020, 07:55 AM
I wish did fix the issue. The estimator parameter wants an object, not a pointer. Currently you are passing a pointer to method which generates the pipeline object. Try adding () to it to solve this:
griglia2 = RandomizedSearchCV(estimator=create_pipeline(), param_distributions = my_param_grid, n_iter=8 )
code :
my_param_grid = dict(dl__hidden=my_neurons, dl__epochs=my_epochs,
                     dl__batch_size=my_batch_size)
my_param_grid = dict(dl__hidden=my_neurons, dl__epochs=my_epochs,
                     dl__batch_size=my_batch_size,
                     scaler__with_mean=False)
tensorflow estimator custom metrics: use sklearn metrics?

tensorflow estimator custom metrics: use sklearn metrics?


By : Jona
Date : March 29 2020, 07:55 AM
this one helps. One solution is to use the metric functions from https://github.com/guillaumegenthial/tf_metrics
Related Posts Related Posts :
  • How to exit/terminate a job earlier and handle the raised exception in apscheduler?
  • python, print intermediate values while loop
  • python to loop over yaml config
  • D3.js is not recognized by PyCharm
  • Access the regularization paths obtained from ElasticNetCV in sklearn
  • Pattern table to Pandas DataFrame
  • Get the earliest date from a column (Python Pandas) after csv.reader
  • Get SystemError: Parent module '' not loaded, cannot perform relative import when trying to import numpy in a Cython Ext
  • Bash or Python : Append and prepend a string recursively in all .tex files
  • Changing a certain index of boolean list of lists change others, too
  • complex dataframe filtering request on the last occurence of a value in Panda/Python [EDIT]
  • How to repeatedly get the contents of a Text widget every loop with tkinter?
  • How to call the tornado.queues message externally
  • How can I use regex in python so that characters not included are disallowed?
  • Discarding randmly scattered empty spaces in pandas data frame
  • Get sums grouped by date by same column filtered by 2 conditions
  • Element disappears when I add an {% include %} tag inside my for loop
  • Django Rest Framework with either a slug or a pk lookup field for the DetailAPIView
  • Flask doesn't stream on Lambda
  • Generate all permutations of fixed length where the elements come from two different sets
  • Making function for calculating distance
  • How to handle multiprocessing based on the limit of CPU's
  • Django - static files is not working
  • Remove x axis and y axis black lines with matplotlib
  • tkinter: assigning multiple functions to one button
  • flask-jwt-extended: Fake Authorization Header during testing (pytest)
  • Setting pandas dataframe value based on row and column conditions
  • swig char ** as a pointer to a char *
  • Confusion over `a` and `b` attributes from scipy.stats.uniform
  • How can I do groupy.apply() without sort my index?
  • Querying Google Cloud datastore with ancestor not returning anything
  • Read value from one thread in Python: queue or global variable?
  • Django - context process query being repeated 102 times
  • Convert a list of images and labels to np array to train tensorflow
  • Lambda not supporting NLTK file size
  • Numpy ndarray image pixel mean for pixel values greater than zero: Normalizing image
  • Understanding output of np.corrcoef for two matrices of different sizes
  • Finding longest perfect match between two strings
  • what is wrong with my cosine similarity? Tensorflow
  • How to manage user content in django?
  • Receiving unsupported operand error while comparing random number and user input.
  • How to wrap the process of creating start_urls in scrapy?
  • How to mark 'duplicated sequence' in pandas?
  • Boolean indexing on multidimensionnal array
  • Unmodified column name index in patsy
  • Cleaner way to unpack nested dictionaries
  • Importing a python module to enable a script to be run from command line
  • Maya Python read and set optionMenu value via variable
  • How can I bind a property to another property in Kivy?
  • Python extracting specific line in text file
  • How to implement n-body simulation with pymunk?
  • Python / matplotlib: print to resolution and without white space / borders / margins
  • Sum up the second value from one dictionary with all values from another dictionary
  • Robot Framework: Open a chrome browser without launching URL
  • Generate inline Bokeh scatterplots in Jupyter using a for loop
  • Group list of dictionaries python
  • Efficient way to apply multiple Boolean mask to set values in a column using pandas
  • Lazy evaluation of a Python dictionary
  • id of xpath is getting changed every time in selenium python 2.7 chrome
  • Matplotlib RuntimeWarning displaying a 3D plot
  • shadow
    Privacy Policy - Terms - Contact Us © voile276.org