gradient-descent

来源： https://stackoverflow.com/questions/63666436/how-to-write-cost-function-formula-from-andrew-ng-assignment-in-octave

来源： https://stackoverflow.com/questions/61980124/full-batch-gradient-descent-with-tf-gradienttape-oom-error

问题 So I am relatively new to the ML/AI game in python, and I'm currently working on a problem surrounding the implementation of a custom objective function for XGBoost. My differential equation knowledge is pretty rusty so I've created a custom obj function with a gradient and hessian that models the mean squared error function that is ran as the default objective function in XGBRegressor to make sure that I am doing all of this correctly. The problem is, the results of the model (the error

问题 So I am relatively new to the ML/AI game in python, and I'm currently working on a problem surrounding the implementation of a custom objective function for XGBoost. My differential equation knowledge is pretty rusty so I've created a custom obj function with a gradient and hessian that models the mean squared error function that is ran as the default objective function in XGBRegressor to make sure that I am doing all of this correctly. The problem is, the results of the model (the error

问题 So I am relatively new to the ML/AI game in python, and I'm currently working on a problem surrounding the implementation of a custom objective function for XGBoost. My differential equation knowledge is pretty rusty so I've created a custom obj function with a gradient and hessian that models the mean squared error function that is ran as the default objective function in XGBRegressor to make sure that I am doing all of this correctly. The problem is, the results of the model (the error

问题 What is the way of implementing Batch gradient descent using sklearn for classification? We have SGDClassifier for Stochastic GD which will take single instance at a time and Linear/Logistic Regression which uses normal equation. 回答1: The possible answer to the question as pointed out in the other similar question as well from sklearn docs: SGD allows minibatch (online/out-of-core) learning, see the partial_fit method. But is partial_fit really a batch gradient decent ? SGD: The gradient of

问题 What is the correct way to perform gradient clipping in pytorch? I have an exploding gradients problem, and I need to program my way around it. 回答1: clip_grad_norm (which is actually deprecated in favor of clip_grad_norm_ following the more consistent syntax of a trailing _ when in-place modification is performed) clips the norm of the overall gradient by concatenating all parameters passed to the function, as can be seen from the documentation: The norm is computed over all gradients

问题 As a self study exercise I am trying to implement gradient descent on a linear regression problem from scratch and plot the resulting iterations on a contour plot. My gradient descent implementation gives the correct result (tested with Sklearn) however the gradient descent plot doesn't seem to be perpendicular to the contour lines. Is this expected or did I get something wrong in my code / understanding? Algorithm Cost function and gradient descent import numpy as np import pandas as pd from

问题 I am reading the a deep learning with python book. After reading chapter 4, Fighting Overfitting, I have two questions. Why might increasing the number of epochs cause overfitting? I know increasing increasing the number of epochs will involve more attempts at gradient descent, will this cause overfitting? During the process of fighting overfitting, will the accuracy be reduced ? 回答1: I'm not sure which book you are reading, so some background information may help before I answer the

问题 I am working on a medical dataset where I am trying to have as less false negatives as possible. A prediction of "disease when actually no disease" is okay for me but a prediction "no disease when actually a disease" is not. That is, I am okay with FP but not FN . After doing some research, I found out ways like Keeping higher learning rate for one class , using class weights , ensemble learning with specificity/sensitivity etc. I achieved the near desired result using class weights like