postgresql-copy

COPY with dynamic file name

老子叫甜甜 提交于 2020-04-18 05:46:00
问题 I am trying to write a function to load csv data into a table. I want the input argument to be the path to the file. CREATE OR REPLACE FUNCTION public.loaddata(filepathname varchar) RETURNS void AS $BODY$ BEGIN COPY climatedata( climatestationid, date, prcp, prcpqflag, prcpmflag, prcpsflag, tmax, tmaxqflag, tmaxmflag, tmaxsflag, tmin, tminqflag, tminmflag, tminsflag) FROM $1 WITH csv header; END; $BODY$ LANGUAGE plpgsql VOLATILE COST 100; ALTER FUNCTION public.filltmaxa(character varying)

Split a big postgres table into multiple csv

旧巷老猫 提交于 2020-02-24 06:49:50
问题 I am using following psql query to connect to a remote host and split a big table into multiple csv files. psql -h xx -p xx -U xx -d xx -c "\COPY (select * from table) TO program 'split --lines 1000' (format csv) I am not getting what mistake I am making here. 回答1: Have you tried using STDOUT ? $ psql -d db -c "COPY (SELECT * FROM t) TO STDOUT CSV " | split -1000 来源: https://stackoverflow.com/questions/60187391/split-a-big-postgres-table-into-multiple-csv

Error on Dynamic csv file export using plpgsql copy to csv in a function

拈花ヽ惹草 提交于 2020-02-07 03:41:44
问题 I am trying to filter a postgresql table for rows that have a product id as a foreign key. For each product id, I need to export 1 csv each to a folder eg , prod1.csv, prod2.csv etc. I have tried to create the function below to automate this but the function is failing when I run it. Can anyone kindly assist me to fix the function or recommend a better approach? CREATE or replace FUNCTION exportdata() RETURNS SETOF record AS $$ DECLARE rec text; BEGIN FOR rec IN ( Select distinct t.products

Execute COPY command in remote database using local file

余生颓废 提交于 2019-12-24 03:47:09
问题 I'm trying to execute the Postgres COPY command. COPY warehouse_terminal.test_table FROM 'C:\file.csv' DELIMITERS E'\t' CSV HEADER QUOTE E'\"' ESCAPE E'\\' NULL AS ''; The problem is, the db where I'm going to use this is remote, and the file "file.csv" is in my local machine in C:\ . As a result, the file is not read. Remote db, but local file. How can I get this working? 回答1: Use \copy of the psql client on your local machine, it's a wrapper for COPY for just this purpose. More details:

How to use \copy in postgresql with pgadmin4

主宰稳场 提交于 2019-12-23 19:08:08
问题 I want to use the \copy command to make csv file with relative path. I used a query to make from test table to csv file named test.csv \copy (SELECT * FROM test) to './test.csv' with csv But in postgresql pgadmin4, it shows that \copy command as a syntax error (there is an underline under the word '\copy') and shows a message like below. ERROR: syntax error at or near "/" LINE 2: /copy (SELECT * FROM test) to './persons_client.csv' with cs... ^ ********** Error ********** ERROR: syntax error

copy data from csv to postgresql using python

邮差的信 提交于 2019-12-20 12:34:31
问题 I am on windows 7 64 bit. I have a csv file 'data.csv'. I want to import data to a postgresql table 'temp_unicommerce_status' via a python script. My Script is: import psycopg2 conn = psycopg2.connect("host='localhost' port='5432' dbname='Ekodev' user='bn_openerp' password='fa05844d'") cur = conn.cursor() cur.execute("""truncate table "meta".temp_unicommerce_status;""") cur.execute("""Copy temp_unicommerce_status from 'C:\Users\n\Desktop\data.csv';""") conn.commit() conn.close() I am getting

Delete rows of a table specified in a text file in Postgres

允我心安 提交于 2019-12-19 09:42:43
问题 I have a text file containing the row numbers of the rows that should be deleted in my table like this: 3 32 40 55 [...] How can I get a PostgreSQL compatible SQL statement which deletes each of these rows from my table using the text file? 回答1: Doing it once could look like this: CREATE TEMP TABLE tmp_x (nr int); COPY tmp_x FROM '/absolute/path/to/file'; DELETE FROM mytable d USING tmp_x WHERE d.mycol = tmp_x.nr; DROP TABLE tmp_x; For repeated use, wrap it into a plpgsql function with file

How to convert date strings to timestamp without knowing the date format

拜拜、爱过 提交于 2019-12-18 14:53:38
问题 I am trying to write a query to insert a value into a timestamp with no timezone data type field. The value is coming from CSV file. The version I am working with is PostgreSQL 8.1.21 . The CSV file upload is done by the client and it has a date column. The date sometimes comes as '28-Sep-13' and sometimes as '28/09/2013' formats. I tried to use the following to cast the string into timestamp: str_date::timestamp . This works fine if str_date is something like '28-Sep-13' but it won't work if

How does COPY work and why is it so much faster than INSERT?

徘徊边缘 提交于 2019-12-18 11:46:14
问题 Today I spent my day improving the performance of my Python script which pushes data into my Postgres database. I was previously inserting records as such: query = "INSERT INTO my_table (a,b,c ... ) VALUES (%s, %s, %s ...)"; for d in data: cursor.execute(query, d) I then re-wrote my script so that it creates an in-memory file than is the used for Postgres' COPY command, which lets me copy data from a file to my table: f = StringIO(my_tsv_string) cursor.copy_expert("COPY my_table FROM STDIN

Problems while importing a txt file into postgres using php

旧街凉风 提交于 2019-12-18 04:23:32
问题 I am trying to import a txt/csv file into my postgres database from php using "\copy" command. I cannot use COPY instead of \copy as I need it to execute as a psql client. My code is: $query = '\\'.'copy data1 FROM "data1.txt" WITH CSV HEADER DELIMITER AS "," QUOTE AS "^"'; $result = pg_query($conn,$query); if (!$result) { echo "cannot copy data\n"; } else { echo "SUCCESS!"; } When I run this php file, I get this error: PHP Warning: pg_query(): Query failed: ERROR: syntax error at or near "\"