MySQL Prepared Statement - How to Loop Through

Question

I have a table filled with ID numbers, which I need to loop through, and use as variables in a prepared statement. I do not know if I need to use a stored procedure for this, o

Answer 1

As others have already suggested, we typically avoid looping through a resultset RBAR (row by agonizing row) primarily for performance reasons. We just don't want to get into the habit of looping through a resultset. But that doesn't answer the question you asked.

To answer the question you asked, here's a rudimentary example of a MySQL stored program that uses a CURSOR to individually process rows returned by a query. MySQL doesn't support anonymous blocks, so the only way to do this is in a MySQL stored program, like a PROCEDURE

DELIMITER $$

CREATE PROCEDURE loop_through_var_list
BEGIN
   DECLARE done INT DEFAULT 0;
   DECLARE v_id INT DEFAULT NULL;  
   DECLARE csr_var_list CURSOR FOR SELECT id FROM var_list ORDER BY id;
   DECLARE CONTINUE HANDLER FOR NOT FOUND SET done = 1;
   OPEN csr_var_list;
   get_id: LOOP
      FETCH csr_var_list INTO v_id; 
      IF done = 1 THEN
         LEAVE get_id;
      END IF;

      -- at this point, we have an id value in v_id, so we can do whatever
      SET @s1 = CONCAT('SELECT ... WHERE id =''', v_id, ''' ...');


   END LOOP get_id;
   CLOSE csr_var_list;
END$$

DELIMITER ;

To execute the procedure:

CALL loop_through_var_list();

NOTES: The syntax for processing a CURSOR in MySQL is quite a bit different than other databases.

To get the "looping", we need to use a LOOP ... END LOOP construct.

But to prevent that loop from running forever, we need a LEAVE statement that will allow us to exit the loop.

We use a conditional test to determine when to leave. In this example, we want to exit after we've finished processing the last row.

The FETCH is going to throw an exception when there are no more rows to be fetched.

We "catch" that exception in a CONTINUE HANDLER (for some arcane reason, the "handlers" have to the last things declared; MySQL throws an error if we try to declare something after a HANDLER (other than another HANDLER.)

When MySQL throws the "no more rows" exception, that fires the handler code. In this example, we're just setting a variable (named done) to a value.

Since it's a "continue" handler, processing starts back up at the statement where the exception was thrown, in this case, that will be the statement following the FETCH. So, the first thing we do is check if we're "done" or not. If we're "done", then we exit the loop, and close the cursor.

Otherwise, we know that we've got an id value from var_list stored in a procedure variable named v_id. So now we can do whatever we want. Looks like you want to put some SQL text into a user-defined variable (including in the value of v_id into the SQL text, then PREPARE, EXECUTE, and DEALLOCATE PREPARE.

Be sure to declare the v_id variable with the appropriate datatype, that matches the datatype of the id column in var_list, I've just assumed that it's an INT.

When we reach the end of the loop, MySQL "loops" back to the beginning of the loop, and off we go again.

In the body of the loop, likely you'll want to CONCAT v_id into the SQL text you want to execute. Looks like you already have a handle on the PREPARE, DEALLOCATE prepare. For testing, you may want to add a LIMIT clause on the SELECT in the cursor declaration, and then do a simple SELECT v_id; in the body, just to verify the loop is working, before you add more code.

---

FOLLOWUP

I wanted to mention another alternative approach to the task, i.e. running a series of statements based on a template, substituting in values provided from a single SQL select statement...

For example, if I had this template:

SELECT * 
  INTO OUTFILE '/tmp/orders_@ID.csv'
  FIELDS TERMINATED BY ',' ENCLOSED BY '"'
  LINES TERMINATED BY '\n'
FROM data 
WHERE id = @ID
ORDER BY 1 

and I needed to replace the occurrences of @ID with a specific id value from a list returned from a SELECT statement, e.g.

SELECT id
  FROM var_list
 WHERE id IS NOT NULL
 GROUP BY id

I probably would not use a MySQL stored program with a CURSOR loop, I'd use a different approach.

I'd make use of a SELECT statement to generate a set of SQL statements which could be executed. Assuming id is integer type, I would likely do something like this:

SELECT CONCAT(' SELECT * 
                   INTO OUTFILE ''/tmp/orders_',s.id,'.csv''
                   FIELDS TERMINATED BY '','' ENCLOSED BY ''"''
                   LINES TERMINATED BY ''\n''
                FROM data
               WHERE id = ',s.id,'
               ORDER BY 1;') AS `stmt`
 FROM ( SELECT v.id
          FROM var_list v
         WHERE v.id IS NOT NULL
         GROUP BY v.id
      ) s
ORDER BY s.id

For every value of id returned from s, the statement returns the text of a SQL SELECT statement that could (and need to) execute. Capturing that into a text file would give me a SQL script I could run.

Answer 2

Use a simple query like:

SELECT *
FROM data
WHERE id IN (
    SELECT id
    FROM var_list
)