load-data-infile

Using MySQL LOAD DATA INFILE with nonprintable character delimiters

本小妞迷上赌 提交于 2019-12-06 05:09:46
I have some vendor data that has the SOH (ASCII character 1) as the field delimiter and STX (ASCII character 2) as the record delimiter. Is it possible to load this data with LOAD DATA INFILE without pre-processing the file and replacing those characters with something more common? I got it. LOAD DATA LOCAL INFILE 'myfile.txt' INTO TABLE my_table CHARACTER SET UTF8 FIELDS TERMINATED BY X'01' LINES TERMINATED BY X'02' (col1, col2, col3); You might try FIELDS TERMINATED BY _ascii 0x02 . I don't know if it will work for LOAD DATA INFILE , but it works in SELECT (i.e., SELECT _ascii 0x61 yields 'a

Load XML Update Table--MySQL

試著忘記壹切 提交于 2019-12-06 04:33:16
LOAD XML LOCAL INFILE 'file1.xml' INTO TABLE my_table ROWS IDENTIFIED BY '<product>'" Is it possible to use this function to update a table? I used REPLACE INTO TABLE my_table but that only added new rows and it did not update the existing rows. LOAD XML LOCAL INFILE 'file1.xml' REPLACE INTO TABLE my_table ROWS IDENTIFIED BY '<product>'" See: http://dev.mysql.com/doc/refman/5.5/en/load-xml.html Note that: REPLACE works exactly like INSERT, except that if an old row in the table has the same value as a new row for a PRIMARY KEY or a UNIQUE index, the old row is deleted before the new row is

Is there a way to populate a MySQL 5.7 table with data from a json file with line breaks on Win7?

自作多情 提交于 2019-12-06 01:38:18
I have following example json file and I'm trying to populate a MySQL table with it. Since MySQL 5.7 supports json as a native datatype I thought this shouldn't be a problem, but MySQL's json validator seems to have a problem with the line breaks inside the file. As soon as I get rid of all line breaks and write my example file in one line, it works perfectly. [ { "somestuff": [ { "field1": "val1", "field2": 17, "field3": 27, "field4": 42, "field5": 73 }, { "field1": "val2", "field2": 3, "field3": 12, "field4": 13, "field5": 100 } ], "field0": "image", "path": "some path" } ] I know json

MYSQL: Display Skipped records after LOAD DATA INFILE?

谁说我不能喝 提交于 2019-12-05 13:33:42
问题 In MySQL I've used LOAD DATA LOCAL INFILE which works fine. At the end I get a message like: Records: 460377 Deleted: 0 Skipped: 145280 Warnings: 0 How can I view the line number of the records that were skipped? SHOW warnings doesn't work: mysql> show warnings; Empty set (0.00 sec) 回答1: If there was no warnings, but some rows were skipped, then it may mean that the primary key was duplicated for the skipped rows. The easiest way to find out duplicates is by openning the local file in excel

How to convert date in .csv file into SQL format before mass insertion

风流意气都作罢 提交于 2019-12-05 12:05:21
I have a csv file with a couple thousand game dates in it, but they are all in the MM/DD/YYYY format 2/27/2011,3:05 PM,26,14 (26 and 14 are team id #s), and trying to put them into SQL like that just results in 0000-00-00 being put into the date field of my table. This is the command I tried using: LOAD DATA LOCAL INFILE 'c:/scheduletest.csv' INTO TABLE game FIELDS TERMINATED BY ',' ENCLOSED BY '"' LINES TERMINATED BY '\n' (`date`, `time`, `awayteam_id`, `hometeam_id`); but again, it wouldn't do the dates right. Is there a way I can have it convert the date as it tries to insert it? I found

Importing CSV data using PHP/MySQL - Mysqli syntax

拥有回忆 提交于 2019-12-05 04:43:21
问题 IN THE BOTTOM OF THIS QUESTION THE FINAL CODE THAT FINALLY WORKED! Trying to implement this (Importing CSV data using PHP/MySQL). I must be almost there... notes1: my $sql came straight from copy/paste phpmyadmin (generate php code) and ran just fine in the phpmyadmin. note2: If I comment the line $sql="DELETE FROM dbase" the code runs just fine (and the table is cleaned). So if i know my sql is right and my code can run other sqls, why does the below does not run?! Im getting: Call to a

What mysql settings affect the speed of LOAD DATA INFILE?

白昼怎懂夜的黑 提交于 2019-12-05 00:26:34
问题 Let me set up the situation. We are trying to insert a modestly high number of rows (roughly 10-20M a day) into a MyISAM table that is modestly wide: +--------------+--------------+------+-----+---------+-------+ | Field | Type | Null | Key | Default | Extra | +--------------+--------------+------+-----+---------+-------+ | blah1 | varchar(255) | NO | PRI | | | | blah2 | varchar(255) | NO | PRI | | | | blah3 | varchar(5) | NO | PRI | | | | blah4 | varchar(5) | NO | PRI | | | | blah5 | varchar

An exception occurred while executing 'LOAD DATA LOCAL INFILE

元气小坏坏 提交于 2019-12-04 21:48:44
I have all previleges to the mysql user for the database but still getting below error [Doctrine\DBAL\DBALException] An exception occurred while executing 'LOAD DATA LOCAL INFILE '/tmp/uk_insiders/tfo.uktrades_transaction_code.out' IGNORE INTO TABLE uktrades_transaction_codes FIELDS TERMINATED BY '|' (id,description)': Warning: PDOStatement::execute(): LOAD DATA LOCAL INFILE forbidden in /Symfony/vendor/doctrine/dbal/lib/Doctrine/DBAL/Statement.php line 138 [Symfony\Component\Debug\Exception\ContextErrorException] Warning: PDOStatement::execute(): LOAD DATA LOCAL INFILE forbidden in /Symfony

How to see progress of .csv upload in MySQL

自古美人都是妖i 提交于 2019-12-04 19:29:33
问题 I have a very large .csv file, and I'm loading it into mysql with the LOAD DATA INFILE command. Because it takes so long, I'd like to see how far along the upload has progressed. I've tried 2 methods so far- First I simply did a SELECT COUNT(*) command to see how many rows had been inserted as the upload was in progress, but that always returns a count of 0. Second, I tried SHOW PROCESSLIST and saw simply how long the query has been running. sometimes the status says 'freeing data' or

Problems with MySQL LOAD XML INFILE

落花浮王杯 提交于 2019-12-04 10:50:48
I have a XML document in the format of... <?xml version="1.0" encoding="UTF-8"?> <yahootable> <row> <various><![CDATA[ multiline text, "&" other <stuff> ]]> </various> <id>1</id> <message><![CDATA[ sdfgsdfg dsfsdfsd ]]> </message> </row> <yahootable> ...and want to use MySQL's LOAD XML LOCAL INFILE to insert it into a table with columns; (various, id, message). I can't seem to get any data from the unparsed CDATA tags into the database columns. Is it that the data between CDATA tags is completely ignored, or is there something I've missed? I was expecting the CDATA would just escape the