fread

Is there a faster way than fread() to read big data?

£可爱£侵袭症+ 提交于 2021-02-18 11:27:28
问题 Hi first of all I already search on stack and google and found posts such at this one : Quickly reading very large tables as dataframes. While those are helpfull and well answered, I'm looking for more informations. I am looking for the best way to read/import "big" data that can go up to 50-60GB. I am currently using the fread() function from data.table and it is the function that is the fastest I know at the moment. The pc/server I work on got a good cpu (work station) and 32 GB RAM, but

Why does an fread loop require an extra Ctrl+D to signal EOF with glibc?

感情迁移 提交于 2021-02-18 10:13:37
问题 Normally, to indicate EOF to a program attached to standard input on a Linux terminal, I need to press Ctrl+D once if I just pressed Enter, or twice otherwise. I noticed that the patch command is different, though. With it, I need to press Ctrl+D twice if I just pressed Enter, or three times otherwise. (Doing cat | patch instead doesn't have this oddity. Also, If I press Ctrl+D before typing any real input at all, it doesn't have this oddity.) Digging into patch 's source code, I traced this

ifstream read and fread not returning same data, C++

拟墨画扇 提交于 2021-02-17 05:24:26
问题 My problem is that using ifstream read and fread on a file descriptor don't seem to produce the same results. I open a file and read its input using ifstream open/read in ios::binary mode. Then I write this buffer out to a file. out1. Next, I open the same file, read its input using FILE* file descriptors and fread. Then I write this buffer out to another file, out2. When I compare out1 to out2 they do not match. out2, which uses FILE*, seems to stop reading, near the end. More worrisome is

How can i check if binary file's content is found in other binary file?

大憨熊 提交于 2021-02-08 08:16:21
问题 i need to check if content in a binary file in in other binary file. i've tried to copy both files content into a array of chars with fread and check them with strstr, but strstr is always returning NULL even if the content supposed to be found in the other file. Any ideas? Thanks. 回答1: Since the strstr function won't work here for an arbitrary binary data (it is working only for strings with \0 . termination), I can see three approaches here: 1) Naive approach: iterate over one array of

How can i check if binary file's content is found in other binary file?

给你一囗甜甜゛ 提交于 2021-02-08 08:11:14
问题 i need to check if content in a binary file in in other binary file. i've tried to copy both files content into a array of chars with fread and check them with strstr, but strstr is always returning NULL even if the content supposed to be found in the other file. Any ideas? Thanks. 回答1: Since the strstr function won't work here for an arbitrary binary data (it is working only for strings with \0 . termination), I can see three approaches here: 1) Naive approach: iterate over one array of

read file with single line into R in fread or read.table

允我心安 提交于 2021-02-08 04:59:24
问题 I want to read a txt file into R this file has only file line, like this 1 NYC 2013-12-30 82 PM2.5 Ⅱ fair 2 London 2013-12-30 66 PM10.0 Ⅱ good there're no \n in this file, and all the what I want is a data.table incorporating these information like this: 1 NYC 2013-12-30 82 PM2.5 Ⅱ fair 2 London 2013-12-30 66 PM10.0 Ⅱ good Luckily, there're no NAs in the file, also I know for sure there're 7 fields for each observations. Could I achieve this using fread? or read.table? I tried this test <-

fread() a struct in c

 ̄綄美尐妖づ 提交于 2021-02-07 06:44:52
问题 For my assignment, I'm required to use fread/fwrite. I wrote #include <stdio.h> #include <string.h> struct rec{ int account; char name[100]; double balance; }; int main() { struct rec rec1; int c; FILE *fptr; fptr = fopen("clients.txt", "r"); if (fptr == NULL) printf("File could not be opened, exiting program.\n"); else { printf("%-10s%-13s%s\n", "Account", "Name", "Balance"); while (!feof(fptr)) { //fscanf(fptr, "%d%s%lf", &rec.account, rec.name, &rec.balance); fread(&rec1, sizeof(rec1),1,

fread why returns 1 for file starting with ff fe read in binary mode

泪湿孤枕 提交于 2021-02-05 09:32:13
问题 I'm trying to read UTF file and decided to read it in binary mode and skip non-ASCII as file consists of valid english text basically. I'm stuck at fread returning 1 instead of number of bytes requested. First output of print_hex IMHO shows it has read more than 1 char. I've read some examples of reading binary files in C e.g Read and write to binary files in C?, read about fread e.g. here https://en.cppreference.com/w/c/io/fread and here How does fread really work?, still puzzled why it

Read csv file with selected rows using data.table's fread

喜你入骨 提交于 2021-01-29 17:36:01
问题 I was going through some earlier post- Quickest way to read a subset of rows of a CSV One way to select subset of data is write.csv(iris,"iris.csv") fread("shuf -n 5 iris.csv") However I was wondering if I can pass some SQL query instead of top 5 rows e.g. only import those rows that have V6 = versicolor Is there any way to do this using fread function? 回答1: This worked for me in windows (unix alternative is grep ) write.csv(iris,"iris.csv") fread(cmd = paste('findstr', 'versicolor', 'iris

fread isnt reading an updated value despite the file has been changed by another program

眉间皱痕 提交于 2021-01-27 16:40:06
问题 There are two programs that are working with my special file at the same time. They have a common beginning: #define TASK_POSITION 0x0100 #include <stdio.h> #include <stdlib.h> #include <inttypes.h> int main() { FILE * control_file; unsigned int task_code; fpos_t task_position; control_file = fopen("/home/anonymous/.control", "r+"); fseek(control_file, TASK_POSITION, SEEK_SET); fgetpos(control_file, &task_position); But later they have a very different code 1st program could change a value at