delimited

Processing a large .txt file in python efficiently

心不动则不痛 提交于 2019-12-30 06:33:22
问题 I am quite new to python and programming in general, but I am trying to run a "sliding window" calculation over a tab delimited .txt file that contains about 7 million lines with python. What I mean by sliding window is that it will run a calculation over say 50,000 lines, report the number and then move up say 10,000 lines and perform the same calculation over another 50,000 lines. I have the calculation and the "sliding window" working correctly and it runs well if I test it on a a small

How to obtain max length of fields in huge pipe delimited file

这一生的挚爱 提交于 2019-12-24 23:03:28
问题 I have a pipe delimited file that is too large to open in Excel. I'm trying to import this file into MSSQL using the import wizard in SSMS. Normally when I do this, I open the file in Excel and use an array function =MAX(LEN(An:Annnn)) to get the max length of each column. Then I use that to specify the size of each field in my table. This file is too large to open in Excel and SQL doesn't check all of the data to give an accurate suggestion (I think it's a crazy small sample like 200 records

MySQL select delimited data

僤鯓⒐⒋嵵緔 提交于 2019-12-24 16:55:59
问题 I have inherited a table with information about some groups of people in which one field which contains delimited data, with the results matched to another table. id_group Name ----------------------- 1 2|4|5 2 3|4|6 3 1|2 And in another table I have a list of people who may belong to one or more groups id_names Names ----------------------- 1 Jack 2 Joe 3 Fred 4 Mary 5 Bill I would like to perform a select on the group data which results in a single field containing a comma or space

MySQL select delimited data

房东的猫 提交于 2019-12-24 16:55:04
问题 I have inherited a table with information about some groups of people in which one field which contains delimited data, with the results matched to another table. id_group Name ----------------------- 1 2|4|5 2 3|4|6 3 1|2 And in another table I have a list of people who may belong to one or more groups id_names Names ----------------------- 1 Jack 2 Joe 3 Fred 4 Mary 5 Bill I would like to perform a select on the group data which results in a single field containing a comma or space

Changing a pipe delimited file to comma delimited in VB.net

[亡魂溺海] 提交于 2019-12-24 01:18:47
问题 So I have a set of pipe delimited inputs which are something like this: "787291 | 3224325523" | 37826427 | 2482472 | "46284729|46246" | 24682 | 82524 | 6846419 | 68247 and I am converting them to comma delimited using the code given below: Dim line As String Dim fields As String() Using sw As New StreamWriter("c:\test\output.txt") Using tfp As New FileIO.TextFieldParser("c:\test\test.txt") tfp.TextFieldType = FileIO.FieldType.Delimited tfp.Delimiters = New String() {"|"} tfp

Ways to read only select columns from a file into R? (A happy medium between `read.table` and `scan`?) [duplicate]

百般思念 提交于 2019-12-17 15:16:36
问题 This question already has answers here : Only read selected columns (3 answers) Closed 5 years ago . I have some very big delimited data files and I want to process only certain columns in R without taking the time and memory to create a data.frame for the whole file. The only options I know of are read.table which is very wasteful when I only want a couple of columns or scan which seems too low level for what I want. Is there a better option, either with pure R or perhaps calling out to some

populating a string vector with tab delimited text

守給你的承諾、 提交于 2019-12-13 22:55:43
问题 I'm very new to C++. I'm trying to populate a vector with elements from a tab delimited file. What is the easiest way to do that? Thanks! 回答1: There could be many ways to do it, simple Google search give you a solution. Here is example from one of my projects. It uses getline and read comma separated file (CSV), I let you change it for reading tab delimited file. ifstream fin(filename.c_str()); string buffer; while(!fin.eof() && getline(fin, buffer)) { size_t prev_pos = 0, curr_pos = 0;

mySQL >> Normalizing a comma-delimited field

限于喜欢 提交于 2019-12-13 09:51:46
问题 I currently have a table called "RESOURCES" with a keywords field called "RES_Tags". The "RES_Tags" field contains a comma-delimited list of keywords for each record. I need to normalize this table/field. I have already set up the following tables: TAGS, TAGS_TO_RESOURCES. Please see the schema here: http://sqlfiddle.com/#!9/edac4/1 What is a query that will allow me to parse the keywords in RES_Tags, write them into the TAGS table without creating duplicates and then write a listing in the

Macro to TRANSPOSE comma delimited cell into rows and copy down adjacent cells

心已入冬 提交于 2019-12-13 09:49:00
问题 I have a spreadsheet in which there are rows of data entered which need to be split. Currently this is a manual process, I have provided a link to the workbook when i have split the steps I have being doing into worksheets: https://www.dropbox.com/s/0p3fg94pa61e4su/Example.xlsx?dl=0 When done manually the logical process is to first split by column E (Temp) and F (Location) as these are directly linked to eachother, then insert a blank row underneath so they are separated as shown in the

Tkinter GUI to Convert Fixed Width File to Delimited File

℡╲_俬逩灬. 提交于 2019-12-11 12:45:43
问题 I am writing a converter code for our Data Department to convert fixed width files into delmited files. Normally we use import the file into Excel, use the text import wizard to set the field lengths, and then just save as a csv. However we have run into the limitation where we have started getting files that are millions of records long, and thus cant be imported into Excel. The files do not always have spaces in between the fields, espicially so between value fields like phone numbers or