variance

Finding the dimension with highest variance using scikit-learn PCA

二次信任 提交于 2019-11-30 03:08:57
I need to use pca to identify the dimensions with the highest variance of a certain set of data. I'm using scikit-learn's pca to do it, but I can't identify from the output of the pca method what are the components of my data with the highest variance. Keep in mind that I don't want to eliminate those dimensions, only identify them. My data is organized as a matrix with 150 rows of data, each one with 4 dimensions. I'm doing as follow: pca = sklearn.decomposition.PCA() pca.fit(data_matrix) When I print pca.explained_variance_ratio_ , it outputs an array of variance ratios ordered from highest

Can I “pimp my library” with an analogue of TraversableLike.map that has nicely variant types?

拈花ヽ惹草 提交于 2019-11-29 19:23:41
问题 Suppose I want to add functionality like map to a Scala List , something along the lines of list mapmap f , which applies the function f to each element of list twice. (A more serious example might be implementing a parallel or distributed map, but I don't want to get distracted by details in that direction.) My first approach would be object MapMap { implicit def createFancyList[A](list: List[A]) = new Object { def mapmap(f: A => A): List[A] = { list map { a: A => f(f(a)) } } } } this now

What is the variance of argument types in Scala?

Deadly 提交于 2019-11-29 17:06:25
I've read about Scala having covariant return types for functions . But what about its argument types? What does FunctionX(T1,...,R) have to do with all this? Gabriele Petronella If you look at the documentation for any FunctionX class, you'll see that the return type is co-variant and the argument types are contravariant. For instance, Function2 has the signature: Function2[-T1, -T2, +R] extends AnyRef You can spot the - and + before the type parameters, where - means contravariant and + covariant. This means that given class Animal class Dog extends Animal then Function1[Animal, Dog] <:

Variance annotations in type aliases

佐手、 提交于 2019-11-29 12:04:33
问题 Recently I've noticed that variance annotations can be used in type aliases. Here is example from Predef : type Function[-A, +B] = Function1[A, B] And I started to think, where it could be used. Obviously, you can't change variance to opposite, or make an invariant type to behave as co- or contravariant. Compiler will throw an error, like this scala> type BrokenFunc[+T, -R] = Function1[T, R] <console>:7: error: covariant type T occurs in contravariant position in type [+T, -R]T => R of type

Why is the var() function giving me a different answer than my calculated variance?

拈花ヽ惹草 提交于 2019-11-29 11:43:41
问题 I wasn't sure if this should go in SO or some other .SE, so I will delete if this is deemed to be off-topic. I have a vector and I'm trying to calculate the variance "by hand" (meaning based on the definition of variance but still performing the calculations in R) using the equation: V[X] = E[X^2] - E[X]^2 where E[X] = sum (x * f(x)) and E[X^2] = sum (x^2 * f(x)) However, my calculated variance is different from the var() function that R has (which I was using to check my work). Why is the

Calculating variance of an image python efficiently

萝らか妹 提交于 2019-11-29 06:54:57
I'm working on a project in which need to get the variance of an image. Currently I'm taking 2 approaches (both work but are very slow): Calculating the variance for each pixel individually: This is the code using numpy, varianceMatrix is the output varianceMatrix = np.zeros(im.shape,np.uint8) w = 1 # the radius of pixels neighbors ny = len(im) nx = len(im[0]) for i in range(w,nx-w): for j in range(w,ny-w): sampleframe = im[j-w:j+w, i-w:i+w] variance = np.var(sampleframe) varianceMatrix[j][i] = int(variance) return varianceMatrix Using an existing scipy function: This is the scipy function:

Finding the dimension with highest variance using scikit-learn PCA

倾然丶 夕夏残阳落幕 提交于 2019-11-29 00:44:52
问题 I need to use pca to identify the dimensions with the highest variance of a certain set of data. I'm using scikit-learn's pca to do it, but I can't identify from the output of the pca method what are the components of my data with the highest variance. Keep in mind that I don't want to eliminate those dimensions, only identify them. My data is organized as a matrix with 150 rows of data, each one with 4 dimensions. I'm doing as follow: pca = sklearn.decomposition.PCA() pca.fit(data_matrix)

Rolling variance algorithm

£可爱£侵袭症+ 提交于 2019-11-28 15:17:44
I'm trying to find an efficient, numerically stable algorithm to calculate a rolling variance (for instance, a variance over a 20-period rolling window). I'm aware of the Welford algorithm that efficiently computes the running variance for a stream of numbers (it requires only one pass), but am not sure if this can be adapted for a rolling window. I would also like the solution to avoid the accuracy problems discussed at the top of this article by John D. Cook. A solution in any language is fine. Mike Taylor I've run across this problem as well. There are some great posts out there in

What is the variance of argument types in Scala?

僤鯓⒐⒋嵵緔 提交于 2019-11-28 10:42:04
问题 I've read about Scala having covariant return types for functions . But what about its argument types? What does FunctionX(T1,...,R) have to do with all this? 回答1: If you look at the documentation for any FunctionX class, you'll see that the return type is co-variant and the argument types are contravariant. For instance, Function2 has the signature: Function2[-T1, -T2, +R] extends AnyRef You can spot the - and + before the type parameters, where - means contravariant and + covariant. This

Calculating variance of an image python efficiently

柔情痞子 提交于 2019-11-28 00:27:11
问题 I'm working on a project in which need to get the variance of an image. Currently I'm taking 2 approaches (both work but are very slow): Calculating the variance for each pixel individually: This is the code using numpy, varianceMatrix is the output varianceMatrix = np.zeros(im.shape,np.uint8) w = 1 # the radius of pixels neighbors ny = len(im) nx = len(im[0]) for i in range(w,nx-w): for j in range(w,ny-w): sampleframe = im[j-w:j+w, i-w:i+w] variance = np.var(sampleframe) varianceMatrix[j][i]