memory-mapped-files

问题 for a new project, I have to read the characters of a file (with configurable encoding) to handle the input. As some of these files can be quite large (> 100MB), I wanted to check out the Java nio's ability to memory map files for faster access. However, I was not able to figure out, how I am able to create something "Reader"-like to read from the MappedByteBuffer with the correct charset decoding. To create the MappedByteBuffer, I currently use: RandomAccessFile raFile = new RandomAccessFile

问题 for a new project, I have to read the characters of a file (with configurable encoding) to handle the input. As some of these files can be quite large (> 100MB), I wanted to check out the Java nio's ability to memory map files for faster access. However, I was not able to figure out, how I am able to create something "Reader"-like to read from the MappedByteBuffer with the correct charset decoding. To create the MappedByteBuffer, I currently use: RandomAccessFile raFile = new RandomAccessFile

问题 The following code throws this exception: "The process cannot access the file '\filename' because it is being used by another process." Fair enough, but what's the proper way to close the reader and/or mmf so that the file can be deleted? I would think that MemoryMappedFile would have a close() method or something similar, but it doesn't. Any help would be greatly appreciated. Thanks. mmf = MemoryMappedFile.CreateFromFile(filename, System.IO.FileMode.OpenOrCreate, "myMap" + fileNo.ToString(),

问题 This morning I asked here why my Python code was (a lot) slower then my F# version but I'm wondering whether the F# version can be made faster. Any ideas how I could create a faster version of the below code that reads a sorted list of unique indexes from a binary file with 32-bit integers? Note that I tried 2 approaches, one based on a BinaryReader, the other one based on MemoryMappedFile (and some more on Github). module SimpleRead let readValue (reader:BinaryReader) cellIndex = // set

问题 With nio it is possible to map an existing file in memory. But is it possible to create it only in memory without file on the hard drive ? I want to mimic the CreateFileMapping windows functions which allow you to write in memory. Is there an equivalent system in Java ? The goal is to write in memory in order for another program ( c ) to read it. 回答1: Have a look at the following. A file is created but this might be as close as your going to get. MappedByteBuffer MappedByteBuffer.load()

问题 Not that it's particularly useful, but I'm curious as to why the following works, is it simply because the page still happens to be in memory even after the file is deleted? In which case, if the page is swapped out the data will be lost? #include <iostream> #include <memory> #include <windows.h> int main() { typedef std::unique_ptr<void, decltype(&CloseHandle)> Handle; typedef std::unique_ptr<void, decltype(&UnmapViewOfFile)> View; View v(nullptr, UnmapViewOfFile); { Handle h(CreateFile( L

问题 I have a project and it needs to access a large amount of proprietary data in ASP.NET. This was done on the Linux/PHP by loading the data in shared memory. I was wondering if trying to use Memory Mapped Files would be the way to go, or if there is a better way with better .NET support. I was thinking of using the Data Cache but not sure of all the pitfalls of size of data being saved in the Cache. 回答1: I know this is a bit late, but the .NET 4.0 framework now supports memory-mapped files out

问题 I have a csv file with two columns, holding measurements from an oscilloscope: Model,MSO4034 Firmware Version,2.48 # ... (15 lines of header) ... -5.0000000e-02,-0.0088 -4.9999990e-02,0.0116 -4.9999980e-02,0.006 -4.9999970e-02,-0.0028 -4.9999960e-02,-0.002 -4.9999950e-02,-0.0028 -4.9999940e-02,0.0092 -4.9999930e-02,-0.0072 -4.9999920e-02,-0.0008 -4.9999910e-02,-0.0056 This data I'd like to load into a numpy array. I could use np.loadtxt: np.loadtxt('data.csv', delimiter=',', skiprows=15,

问题 I have a logfile that is written by a 3rd party application and I'd like my application to "read" that log file in real/near-time, parse the new log entries and act upon certain events. My thought was that I could achieve this with a combination of FileSystemWatcher (to signal file changes) and MemoryMappedFile (to continue reading from a certain offset). However, since this is the first time I'm using MemoryMappedFiles I do run into some issues which probably arise from not understanding the

问题 I have a python3 script that operates with numpy.memmap arrays. It writes an array to newly generated temporary file that is located in /tmp : import numpy, tempfile size = 2 ** 37 * 10 tmp = tempfile.NamedTemporaryFile('w+') array = numpy.memmap(tmp.name, dtype = 'i8', mode = 'w+', shape = size) array[0] = 666 array[size-1] = 777 del array array2 = numpy.memmap(tmp.name, dtype = 'i8', mode = 'r+', shape = size) print('File: {}. Array size: {}. First cell value: {}. Last cell value: {}'.\