Mysql group_concat of repeated keys and count of repetition of multiple columns in 1 query ( Query Optimization )

Question

This question is regarding query optimization to avoid multiple call to database via PHP.

So Here is scenario, I have two tables one contains information you can cal

Answer 1

try this one

SELECT 
     key1, key2, info1, info2, 
     SUM(Scount) AS serial_count, GROUP_CONCAT(Skey1, ' ', Skey2) AS serial_ids,
     SUM(Pcount) AS product_data_count, GROUP_CONCAT(Pkey1, ' ', Pkey2) AS product_data_ids 
FROM 
(

   SELECT DISTINCT 
     IF(b.serial  < 10 OR b.product_data IS NOT NULL,a.key1, NULL) AS `key1`,
     IF(b.serial  < 10 OR b.product_data IS NOT NULL,a.key2, NULL) AS `key2`,
     IF(b.serial  < 10 OR b.product_data IS NOT NULL,a.info1, NULL) AS `info1`, 
     IF(b.serial  < 10 OR b.product_data IS NOT NULL,a.info2, NULL) AS `info2`,
     IF(b.serial  < 10,a.key1, NULL) AS `Skey1`,
     IF(b.serial  < 10,a.key2, NULL) AS `Skey2`,
     IF(b.product_data IS NOT NULL,a.key1, NULL) AS `Pkey1`,
     IF(b.product_data IS NOT NULL,a.key2, NULL) AS `Pkey2`,
     IF(b.serial < 10, 1, NULL) AS `Scount`,
     IF(b.product_data IS NOT NULL, 1, NULL) AS `Pcount`
   FROM main_info a INNER JOIN product1 b ON  a.key1 = b.key1 AND a.key2= b.key2

   UNION ALL

   SELECT DISTINCT
     NULL AS `key1`,
     NULL AS `key2`,
     NULL AS `info1`,
     NULL AS `info2`,
     NULL AS `Skey1`,
     NULL AS `Skey2`,
     NULL AS `Pkey1`,
     NULL AS `Pkey2`,
     IF(serial > 9, 1, NULL) AS `Scount`,
     IF(product_data IS NULL, 1, NULL) AS `Pcount`
   FROM product1 WHERE serial > 9 xor product_data IS NULL

) AS sub GROUP BY info1,info2

RESULT (data from question)

+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| key1 | key2 | info1 | info2 | serial_count | serial_ids  | product_data_count | product_data_ids |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| NULL | NULL | NULL  | NULL  | 1            | NULL        | NULL               | NULL             |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| 1    | 2    | 14    | 92    | 1            | 1 2         | 1                  | 1 2              |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| 1    | 3    | 15    | 82    | 3            | 1 3,1 4,1 5 | 3                  | 1 3,1 4,1 5      |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| 1    | 1    | 15    | 90    | 1            | 1 1         | 1                  | 1 1              |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+

RESULT (data from comment)

+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| key1 | key2 | info1 | info2 | serial_count | serial_ids  | product_data_count | product_data_ids |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| NULL | NULL | NULL  | NULL  | 1            | NULL        | 1                  | NULL             |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| 1    | 2    | 14    | 92    | 1            | 1 2         | 1                  | 1 2              |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| 1    | 3    | 15    | 82    | 3            | 1 3,1 4,1 5 | 3                  | 1 3,1 4,1 5      |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| 1    | 1    | 15    | 90    | 1            | 1 1         | 1                  | 1 1              |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| 2    | 4    | 16    | 88    | 1            | 2 4         | 1                  | 2 4              |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| 2    | 1    | 17    | 90    | NULL         | NULL        | 3                  | 2 1,2 2,2 3      |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+

NOTE:

There is something that I can really understand about the base logic behind the question, so answer mainly base on expected result. Such as if group field (info1 and info2) are null, the other result will always null except for serial_count and product_data_count that can be 1 or null, did you really meant to get that? Notice that this answer use another sub query with UNION ALL to satisfy that.

Answer 2

Judging from your quote it seems to me you want to do something like this (SQLfiddle):

SELECT
  m.info1,
  m.info2,
  COUNT(DISTINCT CONCAT(m.key1, ' ', m.key2)) key_count,
  GROUP_CONCAT(DISTINCT CONCAT(m.key1, ' ', m.key2) ORDER BY m.key1, m.key2) key_pairs,
  COUNT(DISTINCT p.serial) serial_count,
  GROUP_CONCAT(DISTINCT p.serial ORDER BY p.serial) serials,
  COUNT(DISTINCT p.product_data) data_count,
  GROUP_CONCAT(DISTINCT p.product_data ORDER BY p.product_data) product_data
FROM
  main_info m INNER JOIN
  product1 p ON p.key1 = m.key1 AND p.key2 = m.key2
WHERE
  p.serial < 10
GROUP BY
  m.info1,
  m.info2

Count distinct values and list them, is this correct? You can't just group by info1, info2 and also have columns for key1 or key2 in the result (e.g. min(key1) or max(key2) would work). I adjusted this in the query above, although it is quite different from your result it might be what you actually need, maybe with a few changes.

Answer 3

How about combining your two queries with a JOIN?

SQL:

 SELECT
    tbl1.key1, tbl1.key2, tbl1.info1, tbl1.info2, tbl1.serial_count, tbl1.serial_ids,
    tbl2.product_data_count, tbl2.product_data_ids
 FROM 
 (
select * , count(*) as serial_count,GROUP_CONCAT(key1,' ',key2) as serial_ids from 
 (
 SELECT distinct 
 if(b.serial  < 10,a.key1,null) AS `key1`,
 if(b.serial  < 10,a.key2,null) AS `key2`,
 if(b.serial  < 10,a.info1,null) AS `info1`, 
         if(b.serial  < 10,a.info2,null) AS `info2`
 FROM main_info a inner join product1 b on  a.key1 = b.key1 AND a.key2= b.key2
 ) as sub group by info1,info2
 ) tbl1
 LEFT OUTER JOIN 
 (
select * , count(*) as product_data_count,GROUP_CONCAT(key1,' ',key2) as product_data_ids from 
 (
 SELECT distinct 
 if(b.product_data IS NOT NULL,a.key1,null) AS `key1`,
 if(b.product_data IS NOT NULL,a.key2,null) AS `key2`,
 if(b.product_data IS NOT NULL,a.info1,null) AS `info1`, 
         if(b.product_data IS NOT NULL,a.info2,null) AS `info2`
 FROM main_info a inner join product1 b on  a.key1 = b.key1 AND a.key2= b.key2
 ) as sub group by info1,info2
 ) tbl2
 ON tbl1.info1 = tbl2.info1 AND tbl1.info2 = tbl2.info2
 ORDER BY 3,4
 ;

Output:

mysql>  SELECT
    -> tbl1.key1, tbl1.key2, tbl1.info1, tbl1.info2, tbl1.serial_count, tbl1.serial_ids,
    -> tbl2.product_data_count, tbl2.product_data_ids
    ->  FROM
    ->  (
    -> select * , count(*) as serial_count,GROUP_CONCAT(key1,' ',key2) as serial_ids from
    ->  (
    ->  SELECT distinct
    ->  if(b.serial  < 10,a.key1,null) AS `key1`,
    ->  if(b.serial  < 10,a.key2,null) AS `key2`,
    ->  if(b.serial  < 10,a.info1,null) AS `info1`,
    ->          if(b.serial  < 10,a.info2,null) AS `info2`
    ->  FROM main_info a inner join product1 b on  a.key1 = b.key1 AND a.key2= b.key2
    ->  ) as sub group by info1,info2
    ->  ) tbl1
    ->  LEFT OUTER JOIN
    ->  (
    -> select * , count(*) as product_data_count,GROUP_CONCAT(key1,' ',key2) as product_data_ids from
    ->  (
    ->  SELECT distinct
    ->  if(b.product_data IS NOT NULL,a.key1,null) AS `key1`,
    ->  if(b.product_data IS NOT NULL,a.key2,null) AS `key2`,
    ->  if(b.product_data IS NOT NULL,a.info1,null) AS `info1`,
    ->          if(b.product_data IS NOT NULL,a.info2,null) AS `info2`
    ->  FROM main_info a inner join product1 b on  a.key1 = b.key1 AND a.key2= b.key2
    ->  ) as sub group by info1,info2
    ->  ) tbl2
    ->  ON tbl1.info1 = tbl2.info1 AND tbl1.info2 = tbl2.info2
    ->  ORDER BY 3,4
    ->  ;
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| key1 | key2 | info1 | info2 | serial_count | serial_ids  | product_data_count | product_data_ids |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
| NULL | NULL |  NULL |  NULL |            1 | NULL        |               NULL | NULL             |
|    1 |    2 |    14 |    92 |            1 | 1 2         |                  1 | 1 2              |
|    1 |    3 |    15 |    82 |            3 | 1 3,1 4,1 5 |                  3 | 1 3,1 4,1 5      |
|    1 |    1 |    15 |    90 |            1 | 1 1         |                  1 | 1 1              |
|    2 |    1 |    17 |    90 |            1 | 2 1         |                  3 | 2 2,2 3,2 1      |
+------+------+-------+-------+--------------+-------------+--------------------+------------------+
5 rows in set (0.01 sec)

mysql>  select version();
+-----------------+
| version()       |
+-----------------+
| 10.1.10-MariaDB |
+-----------------+
1 row in set (0.00 sec)