BigQuery - same query works when submitted from UI and reports SQL syntax error from batch

Question

I have a SQL query involving two joins on different fields.

When I run this query interactively in the UI, I get back a result set, no problem. When I submit the exa

Answer 1

The difference between the succeeding and the failing query appears to be that you are setting flattenResults=false when you run the query in batch. This mode has slightly different behavior with JOINs that can cause subtle issues like this one.

From BigQuery docs on JOINs:

BigQuery executes multiple JOIN operations pairwise, starting with the first pair of inputs after the FROM keyword. Subsequent JOIN operations use the results of the previous JOIN operation as the left JOIN input.

The underlying issue here is that the left source of the final JOIN in your query is trying to collapse t1.video and t2.video into the same scope, both with the name "video", and causing this error.

A simple reproducer for this issue is the following, with flattenResults=false:

SELECT t1.video
FROM (SELECT 17 as video, 18 as session) t1
JOIN (SELECT 17 as video) t2
ON t1.video = t2.video
JOIN (SELECT 18 as session) t3
ON t1.session = t3.session

You have a couple options to make this query work:

1. Stop setting flattenResults=false. I might be wrong, but from glancing at your query, it didn't look like it would return any nested or repeated fields, so it may not be necessary.

2. Rename one of the video fields in either t1 or t2. e.g.:

   SELECT t1.video
   FROM (SELECT 17 as video, 18 as session) t1
   JOIN (SELECT 17 as video2) t2
   ON t1.video = t2.video2
   JOIN (SELECT 18 as session) t3
   ON t1.session = t3.session
   

3. Wrap the first JOIN in a subselect. This allows you to perform the disambiguation by only selecting one of the video fields. e.g.:

   SELECT t1_and_t2.video FROM
     (SELECT t1.video as video, t1.session as session
     FROM (SELECT 17 as video, 18 as session) t1
     JOIN (SELECT 17 as video) t2 
     ON t1.video = t2.video) t1_and_t2
   JOIN (SELECT 18 as session) t3
   ON t1_and_t2.session = t3.session;