Using BigQuery to find outliers with standard deviation results combined with WHERE clause

こ雲淡風輕ζ 提交于 2019-12-13 07:26:29

问题


Standard deviation analysis can be a useful way to find outliers. Is there a way to incorporate the result of this query (finding the value of the fourth standard deviation away from the mean)...

SELECT (AVG(weight_pounds) + STDDEV(weight_pounds) * 4) as high FROM [publicdata:samples.natality];

result = 12.721342001626912

...Into another query that produces information about which states and dates have the most babies born heavier that 4 standard deviations from average?

SELECT state, year, month ,COUNT(*) AS outlier_count
 FROM [publicdata:samples.natality]
WHERE
  (weight_pounds > 12.721342001626912)
AND
  (state != '' AND state IS NOT NULL)
GROUP BY state, year, month 
ORDER BY outlier_count DESC;

Result:

Row  state   year    month   outlier_count    
1    MD  1990    12  22   
2    NY  1989    10  17   
3    CA  1991    9   14

Essentially it would be great to combine this into a single query.


回答1:


You can abuse JOIN for this (and thus performance will suffer):

SELECT n.state, n.year, n.month ,COUNT(*) AS outlier_count
FROM (
  SELECT state, year, month, weight_pounds, 1 as key 
  FROM [publicdata:samples.natality]) as n
JOIN (
  SELECT (AVG(weight_pounds) + STDDEV(weight_pounds) * 4) as giant_baby, 
          1 as key 
  FROM [publicdata:samples.natality]) as o
ON n.key = o.key
WHERE
  (n.weight_pounds > o.giant_baby)
AND
  (n.state != '' AND n.state IS NOT NULL)
GROUP BY n.state, n.year, n.month 
ORDER BY outlier_count DESC;


来源:https://stackoverflow.com/questions/12534218/using-bigquery-to-find-outliers-with-standard-deviation-results-combined-with-wh

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!