tokyo-cabinet | 易学教程

Tokyo Cabinet hash table breaks when reaching 65 GB

阅读更多关于 Tokyo Cabinet hash table breaks when reaching 65 GB

问题 I'm using the Tokyo Cabinet database through the PyTyrant module, storing the data in a hash table (*.tch file). When the file reaches 65 GB in size, the following bug occurs: when a new entry is added to the database, querying the new key gives a "no such key error". However, the new key is listed when I query for a list of all keys. Furthermore, the problematic entries cannot be deleted from the table. Can anyone explain this? EDIT: output of 'tcrmgr inform -st -port 1978 localhost'

Issue with Hash Map Space

阅读更多关于 Issue with Hash Map Space

问题 In my Java code, I am using Guava's Multimap (com.google.common.collect.Multimap) by using this: Multimap<Integer, Integer> Index = HashMultimap.create() Here, Multimap key is some portion of a URL and value is another portion of the URL (converted into an integer). Now, I assign my JVM 2560 Mb (2.5 GB) heap space (by using Xmx and Xms). However, it can only store 9 millions of such (key,value) pairs of integers (approx 10 million). Now, issue is, I can provide JVM only limited amount of

Issue with Hash Map Space

阅读更多关于 Issue with Hash Map Space

Tokyo Cabinet vs SQLite3 on iPhone

阅读更多关于 Tokyo Cabinet vs SQLite3 on iPhone

问题 Has anyone used Tokyo Cabinet on the iPhone? I'm interested to see if there are any real world performance differences between it and SQLite3. Also, SQLite 3 has the expressive power of SQL, does Tokyo Cabinet have any kind of query language? Any input would be greatly appreciated, thanks. 回答1: I have not used either on the iPhone specifically, but I have used both for various projects. Like you pointed out, SQLite does provide SQL query language which means you have much more flexibility in

How to bundle a native library and a JNI library inside a JAR?

阅读更多关于 How to bundle a native library and a JNI library inside a JAR?

问题 The library in question is Tokyo Cabinet. I want is to have the native library, JNI library, and all Java API classes in one JAR file to avoid redistribution headaches. There seems to be an attempt at this at GitHub, but It does not include the actual native library, only JNI library. It seems to be specific to Leiningen's native dependencies plugin (it won't work as a redistributable). The question is, can I bundle everything in one JAR and redistribute it? If yes, how? P.S.: Yes, I realize

Tokyo Cabinet vs SQLite3 on iPhone

阅读更多关于 Tokyo Cabinet vs SQLite3 on iPhone

Has anyone used Tokyo Cabinet on the iPhone? I'm interested to see if there are any real world performance differences between it and SQLite3. Also, SQLite 3 has the expressive power of SQL, does Tokyo Cabinet have any kind of query language? Any input would be greatly appreciated, thanks. I have not used either on the iPhone specifically, but I have used both for various projects. Like you pointed out, SQLite does provide SQL query language which means you have much more flexibility in how you can search for and find data. Assuming you are comfortable with SQL you will be able to easily

Issue with Hash Map Space and Performance

阅读更多关于 Issue with Hash Map Space and Performance

问题 I have to store more than 100 millions of key-values in my HashMultiMap (key can have multiple values). Can anybody help me which one is faster for both storing and searching: 1) Berkeley DB 2) Tokyo Cabinet 3) H2 4) EhCache 5) Or anyothers Another point, is performance of those approximately identical to in-memory hash map ? A little bit guidance will be more helpful. Thanks. NB: information about any one of these is also helpful. 回答1: I'd recommend Redis. It's more of a data structure store

Ruby Rack: startup and teardown operations (Tokyo Cabinet connection)

阅读更多关于 Ruby Rack: startup and teardown operations (Tokyo Cabinet connection)

I have built a pretty simple REST service in Sinatra, on Rack. It's backed by 3 Tokyo Cabinet/Table datastores, which have connections that need to be opened and closed. I have two model classes written in straight Ruby that currently simply connect, get or put what they need, and then disconnect. Obviously, this isn't going to work long-term. I also have some Rack middleware like Warden that rely on these model classes. What's the best way to manage opening and closing the connections? Rack doesn't provide startup/shutdown hooks as I'm aware. I thought about inserting a piece of middleware

Ruby Rack: startup and teardown operations (Tokyo Cabinet connection)

阅读更多关于 Ruby Rack: startup and teardown operations (Tokyo Cabinet connection)

问题 I have built a pretty simple REST service in Sinatra, on Rack. It's backed by 3 Tokyo Cabinet/Table datastores, which have connections that need to be opened and closed. I have two model classes written in straight Ruby that currently simply connect, get or put what they need, and then disconnect. Obviously, this isn't going to work long-term. I also have some Rack middleware like Warden that rely on these model classes. What's the best way to manage opening and closing the connections? Rack

Why does tokyo tyrant slow down exponentially even after adjusting bnum?

阅读更多关于 Why does tokyo tyrant slow down exponentially even after adjusting bnum?

Has anyone successfully used Tokyo Cabinet / Tokyo Tyrant with large datasets? I am trying to upload a subgraph of the Wikipedia datasource. After hitting about 30 million records, I get exponential slow down. This occurs with both the HDB and BDB databases. I adjusted bnum to 2-4x the expected number of records for the HDB case with only a slight speed up. I also set xmsiz to 1GB or so but ultimately I still hit a wall. It seems that Tokyo Tyrant is basically an in memory database and after you exceed the xmsiz or your RAM, you get a barely usable database. Has anyone else encountered this