monetdb

快速读取非常大的表作为数据框

由 ε祈祈猫儿з 提交于 2020-02-26 16:09:15
我有非常大的表（3000万行），我想将其作为R中的数据帧加载 read.table() 具有许多方便的功能，但似乎实现中有很多逻辑会减慢速度事情下来。 就我而言，我假设我提前知道了列的类型，该表不包含任何列标题或行名，并且没有任何我要担心的病理字符。 我知道使用 scan() 以列表的形式读取表的速度非常快，例如： datalist <- scan('myfile',sep='\t',list(url='',popularity=0,mintime=0,maxtime=0))) 但是我将其转换为数据帧的一些尝试似乎使上述性能降低了6倍： df <- as.data.frame(scan('myfile',sep='\t',list(url='',popularity=0,mintime=0,maxtime=0)))) 有更好的方法吗？ 还是完全不同的解决方法？ #1楼 奇怪的是，即使这是很重要的问题，多年来也没有人回答问题的最底层部分 data.frame 只是具有正确属性的列表，因此，如果您有大量数据，则不想将其用作 as.data.frame 或类似的列表。 只需将列表“就地”转换为数据框就可以更快： attr(df, "row.names") <- .set_row_names(length(df[[1]])) class(df) <- "data.frame"

How to display all columns and its data type in a table via SQL query

由 假装没事ソ 提交于 2020-02-06 15:44:45
问题 I am trying to print the column names from a table called 'meta' and I need also its data types. I tried this query SELECT meta FROM INFORMATION_SCHEMA.TABLES; but it throws an error saying no information schema available. Could you please help me, I am a beginner in SQL. Edit: select tables.name from tables join schemas on tables.schema_id=schemas.id where schemas.name=’sprl_db’ ; This query gives me all the tables in database 'sprl_db' 回答1: You can use the monetdb catalog: select c.name, c

MonetDB connect to GO.db within R code that is run in-database

由 泄露秘密 提交于 2020-01-07 02:03:06
问题 I am trying to run some R code in database. Most of it is going pretty well, but I seem to have stumbled on a bug. I cannot load the following package, which is a dependency for some of my code. WGCNA I have been chasing it down, and it seems to be due to an error when trying to load GO.db. I get the following error: Error in .local(conn, statement, ...) : Unable to execute statement 'SELECT * FROM tmp_test();'. Server says '!Error running R expression. Error message: Error in as.data.frame(

Unable to connect R to MonetDB (MonetDB.R)

由 心已入冬 提交于 2020-01-04 19:17:21
问题 I'm trying to start with the MonetDB.R package but I struggle to setup a connection ( dbConnect ). Here is a reproductible example require(MonetDB.R) drv <- dbDriver('MonetDB.R') con <- dbConnect(drv, 'monetdb://localhost/demo') ## Process R aborted (core dumped) at Thu Dec 26 11:03:47 2013 ## R: mapi.c:72: mapiConnect: Assertion `Rf_isInteger(port)' failed. Is there any trick to fix this error ? I'm using the developpement version of MonetDB ( Database: MonetDB v11.18.0 (unreleased) ) , the

What can be reasons for `Error in .local(conn, statement, …)´ in dbWriteTable from package MonetDB.R?

由 十年热恋 提交于 2019-12-24 10:56:11
问题 I get the error, after I set up MonetDB and try to write a large data.frame as a new table in the default database (demo): >dbWriteTable(conn, "table1", df) Error in .local(conn, statement, ...) : Unable to execute statement 'INSERT INTO table1 VALUES([...]) The data.frame has dimensions: >dim(df) [1] 148767 618 And has all columns formatted as character: >all(lapply(df,class)=='character') [1] TRUE The error seems to stem from a string value being too long (Thanks @Hannes Mühleisen):

monetdb sql method to locate or match by the nearest value, without a TOP or LIMIT

由 耗尽温柔 提交于 2019-12-13 19:38:33
问题 i am trying to replicate this question in monetdb, which i don't believe supports the TOP command and can only use LIMIT on the outermost result. if i could use TOP then i believe this command would give me what i want. is there a reasonable alternative that's not massively inefficient? i need to run this on a table that will quickly max out my server's ram. thanks! CREATE TABLE nearest_matches AS ( SELECT a.* , ( SELECT TOP 1 svcmon FROM person_table AS z WHERE a.yr = z.yr AND a.person_id =

Which open source dashboard/BI tools can work with monetDB?

由 这一生的挚爱 提交于 2019-12-13 04:46:49
问题 I'm trying to create a rich online dashboard to analyze web traffic with monetDB. Does anybody know how to integrate with an open source solution ? 回答1: I would recommend to use: DWH: MonetDB ETL: PDI (Pentaho Data Integration) OLAP: Mondrian OLAP (OLAP schema workbench tool) Dashboards: Pentaho BI Server CE (CDF: charts portfolio CCC + maps, etc.) For a quick start: Fill you DWH (MonetDB) with sample data (One fact table, few dimensions) Create OLAP schema in on the top of the DWH tables

Replace an entire field value in a file using awk or other

由 对着背影说爱祢 提交于 2019-12-13 03:47:31
问题 I have an export from postgresql table with multiple fields, including boolean (exported by postgresql as t and f character), and I need to import it in another database (monetdb) that won't understand t/f as bool values. ( EDIT removed spaces to reflect true file aspect and avoid angry comments - previously there was spaces displayed) id|val_str|bool_1|bool2|bool_3|bool4| 1|help|t|t|f|t| 2|test|f|t|f|f| ... As I cannot replace all occurence of t / f I need to integrate the field separator in

Is there a way to view MonetDB Function signatures

由 烈酒焚心 提交于 2019-12-12 03:38:02
问题 I have tried to locate a source for MonetDB function signatures. Some can be found by querying the sys.functions table, but date and time functions are missing the signatures. for instance if you look at the "month" function in that table there are 4 listed in the table. | id | name | func | mod | lang | type | side_e | varres | vararg |> : : : : : uage : : ffect : : :> +======+============+=======+=======+======+======+========+========+======== + | 901 | month | month | mtime | 0 | 1 |

MonetDB client compilation

由 ∥☆過路亽．° 提交于 2019-12-11 17:47:13
问题 I'm a beginner in the Database community. I have been trying to interface with MonetDB using C/C++ MAPI. I have installed MonetDB on my Ubuntu 14.04 LTS machine and when I try to compile a sample program from here, I get the following error: $ gcc test.c -I /usr/include/monetdb -lmapi test.c:(.text+0x29): undefined reference to mapi_explain_query' test.c:(.text+0x35): undefined reference to mapi_result_error' test.c:(.text+0x50): undefined reference to mapi_explain_result' test.c:(.text+0x5c)
订阅 monetdb