snowflake-cloud-data-platform

问题 I have a Pandas dataframe that I'm writing out to Snowflake using SQLAlchemy engine and the to_sql function. It works fine, but I have to use the chunksize option because of some Snowflake limit. This is also fine for smaller dataframes. However, some dataframes are 500k+ rows, and at a 15k records per chunk, it takes forever to complete writing to Snowflake. I did some research and came across the pd_writer method provided by Snowflake, which apparently loads the dataframe much faster. My

问题 I have two related tables (unnecessary columns not listed): LOCATION VENUE_ID - NUMBER(38,0) VISIT ID - NUMBER(38,0) VENUE_ID - NUMBER(38,0) DEVICE_ID - VARCHAR(16777216) The tables are related such that visits are associated with a location via VENUE_ID . I'm attempting to get the count of unique device ids by location, so I wrote the following query: SELECT "d"."VENUE_ID" , ( SELECT COUNT(*) FROM ( SELECT DISTINCT "f0"."DEVICE_ID" FROM "MAIN"."VISIT" AS "f0" WHERE "d"."VENUE_ID" = "f0".

问题 I am using this code to get large xml data in staging into snowflake, for which I have to use STRIP_OUTER_ELEMENT = TRUE, other wise error occurs: Error parsing XML: document is too large, max size 16777216 bytes COPY INTO SAMPLE_DB.SAMPLE_SCH.T_TABLE (CATALOG_XML) FROM @META_DB.CONFIG.STAGESNOWFLAKE/catalogmain.xml FILE_FORMAT=(TYPE=XML STRIP_OUTER_ELEMENT = TRUE) ON_ERROR='CONTINUE'; ON THIS XML, which is very large <catalog xmlns="http://www.demandware.com/xml/impex/catalog/2006-10-31"

问题 I am using this code to get large xml data in staging into snowflake, for which I have to use STRIP_OUTER_ELEMENT = TRUE, other wise error occurs: Error parsing XML: document is too large, max size 16777216 bytes COPY INTO SAMPLE_DB.SAMPLE_SCH.T_TABLE (CATALOG_XML) FROM @META_DB.CONFIG.STAGESNOWFLAKE/catalogmain.xml FILE_FORMAT=(TYPE=XML STRIP_OUTER_ELEMENT = TRUE) ON_ERROR='CONTINUE'; ON THIS XML, which is very large <catalog xmlns="http://www.demandware.com/xml/impex/catalog/2006-10-31"

问题 Run this in Snowflake to see that it returns an error. with a as ( select to_geography('MULTIPOLYGON (((-75.567446994279891 39.5086159918784 0 0, -75.562456 39.51265 0 0, -75.560317002808333 39.515950000160686 0 0, -75.561742999999993 39.520534 0 0, -75.565546 39.514849999999996 0 0, -75.567446994279891 39.5086159918784 0 0)), ((-75.571759009184774 39.623583997285259 0 0, -75.5712463122867 39.622342955203095 0 0, -75.567694 39.613744 0 0, -75.561934 39.605216 0 0, -75.555869991521988 39

问题 Run this in Snowflake to see that it returns an error. with a as ( select to_geography('MULTIPOLYGON (((-75.567446994279891 39.5086159918784 0 0, -75.562456 39.51265 0 0, -75.560317002808333 39.515950000160686 0 0, -75.561742999999993 39.520534 0 0, -75.565546 39.514849999999996 0 0, -75.567446994279891 39.5086159918784 0 0)), ((-75.571759009184774 39.623583997285259 0 0, -75.5712463122867 39.622342955203095 0 0, -75.567694 39.613744 0 0, -75.561934 39.605216 0 0, -75.555869991521988 39

问题 After upgrading to macOS Big Sur my snowsql scripts all fail with "The Library Security could not be found" $ snowsql -q "select 1 x" User: me Password: XXX The library Security could not be found If the error message is unclear, enable logging using -o log_level=DEBUG and see the log to find out the cause. Contact support for further help. Goodbye! This reminds me of an issue I ran into with the snowflake-connector-python library involving oscrypto library being used instead of openssl when

问题 I'm trying to make a SELECT on multiple joined VARIANT columned tables. The main record is returned as DATA and all supporting information around it are made up of supporting joined tables returned as INCLUDED. I'm using an ARRAY_CONSTRUCT_COMPACT on the supporting VARIANT records AND ARRAY_AGG(DISTINCT [record]) to aggregate them and de-duplicate. The issue is that ARRAY_AGG produces multiple records on one of my joined tables (ENTITIES). When they are constructed using ARRAY_CONSTRUCT

来源： https://stackoverflow.com/questions/60176445/can-snowsql-be-configured-to-run-a-login-sql-script-and-not-exit

来源： https://stackoverflow.com/questions/64505552/attributeerror-snowflakecursor-object-has-no-attribute-cursor